# C4W1\_Assignment Exercise 4 - Translator

**URL:** <https://community.deeplearning.ai/t/c4w1-assignment-exercise-4-translator/541423>\
**Category:** NLP with Attention Models\
**Tags:** week-module-1\
**Created:** [January 11, 2024, 8:35am UTC](https://community.deeplearning.ai/t/c4w1-assignment-exercise-4-translator/541423 "2024-01-11T08:35:52Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![sugaprho](https://avatars.discourse-cdn.com/v4/letter/s/df705f/32.png) [@sugaprho](https://community.deeplearning.ai/u/sugaprho)\
**Post date:** [January 11, 2024, 8:35am UTC](https://community.deeplearning.ai/t/c4w1-assignment-exercise-4-translator/541423/1 "2024-01-11T08:35:52Z")

</div>

Hello.

I am having difficulty complete the Exercise 4 - Translator cell.

All previous blocks up to Exercise 3 passed unit test successfully and quick check of your implementation just below Exercise 4 - Translator block matches the Expected Output.

However when I execute `w1_unittest.test_translator(Translator, Encoder, Decoder)` cell, I get the following error messages which are raised by attn\_output = …

The error message is:

InvalidArgumentError: Exception encountered when calling layer ‘key’ (type EinsumDense).

{{function\_node \__wrapped\_\_Einsum\_N\_2\_device_/job:localhost/replica:0/task:0/device:GPU:0}} Expected dimension 512 at axis 0 of the input shaped [256,1,256] but got dimension 256 [Op:Einsum] name:

Call arguments received by layer ‘key’ (type EinsumDense):  
• inputs=tf.Tensor(shape=(64, 15, 512), dtype=float32)

Can anyone give me some advice please?

---

<div class="post-metadata">

**Author:** ![arvyzukai](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/arvyzukai/32/165165_2.png) [@arvyzukai](https://community.deeplearning.ai/u/arvyzukai)\
**Post date:** [January 11, 2024, 10:07am UTC](https://community.deeplearning.ai/t/c4w1-assignment-exercise-4-translator/541423/2 "2024-01-11T10:07:57Z")

</div>

Hi @sugaprho

The reason for this error is clearly dimension mismatch. If I had to guess, you probably hard-coded the 256 value instead of using `units` parameter in this or previous exercises?

---

<div class="post-metadata">

**Author:** ![sugaprho](https://avatars.discourse-cdn.com/v4/letter/s/df705f/32.png) [@sugaprho](https://community.deeplearning.ai/u/sugaprho)\
**Post date:** [January 21, 2024, 2:19am UTC](https://community.deeplearning.ai/t/c4w1-assignment-exercise-4-translator/541423/3 "2024-01-21T02:19:38Z")

</div>

Hi @arvyzukai, I really appreciate for your kind response.  
I didn’t hard code values, but I used parameters in the constructor or call method.  
However, I closely looked at what mistake I made and I finally figure out what was wrong in my source code.

To do so, I printed the shape of data layer by layer.

In the Decoder, the data shapes evolves as follows.

1. Input (64,15)
2. Embedding (64,15,256)
3. LSTM (64,15,256)
4. Attention (64,15,256)
5. LSTM ~~(64,256)~~ → (64, 15, 256)
6. Dense ~~(64,12000)~~ → (64, 15, 12000)

In my case, I made mistake at step 5. After I corrected some parameter, my source pass unit test.

Thank you.

---

<div class="post-metadata">

**Author:** ![Krzysztof\_Jakubczyk](https://avatars.discourse-cdn.com/v4/letter/k/ebca7d/32.png) [@Krzysztof\_Jakubczyk](https://community.deeplearning.ai/u/Krzysztof_Jakubczyk)\
**Post date:** [January 21, 2024, 4:31pm UTC](https://community.deeplearning.ai/t/c4w1-assignment-exercise-4-translator/541423/4 "2024-01-21T16:31:58Z")

</div>

Hi,  
can you share where was the issue? I am facing similar problem, my Decoder tests pass bu the Translator passes the “quick check” with proper output but fails on the unit tests. Looks to me that embedding layer in the decoder produces different output size Here’s the output of input parameters and dimensions in different steps:

```auto

Translator: 10000 512
Encoder: 10000, 512
Decoder: 10000, 512
Translator Context: (64, 15)
Translator Target: (64, 14)
Encoder context: (64, 15)

Encoder embedding: (64, 15, 512)

Encoder LSTM: (64, 15, 512)

Translator: Encoded context: (64, 15, 512)
Decoder input: (64, 15, 512) (64, 14)
Decoder Embedding: (64, 14, 256)
Decoder LSTM: (64, 14, 256)

```

---

<div class="post-metadata">

**Author:** ![Krzysztof\_Jakubczyk](https://avatars.discourse-cdn.com/v4/letter/k/ebca7d/32.png) [@Krzysztof\_Jakubczyk](https://community.deeplearning.ai/u/Krzysztof_Jakubczyk)\
**Post date:** [January 21, 2024, 6:12pm UTC](https://community.deeplearning.ai/t/c4w1-assignment-exercise-4-translator/541423/5 "2024-01-21T18:12:20Z")

</div>

Just for clarity my error is similar:

```auto
InvalidArgumentError: Exception encountered when calling layer 'key' (type EinsumDense).

{{function_node __wrapped__ Einsum_N_2_device_/job:localhost/replica:0/task:0/device:GPU:0}} Expected dimension 512 at axis 0 of the input shaped [256,1,256] but got dimension 256 [Op:Einsum] name: 

Call arguments received by layer 'key' (type EinsumDense):
  • inputs=tf.Tensor(shape=(64, 19, 512), dtype=float32)

```

on calling self.mha in CrossAttention

---

<div class="post-metadata">

**Author:** ![sugaprho](https://avatars.discourse-cdn.com/v4/letter/s/df705f/32.png) [@sugaprho](https://community.deeplearning.ai/u/sugaprho)\
**Post date:** [January 25, 2024, 11:04am UTC](https://community.deeplearning.ai/t/c4w1-assignment-exercise-4-translator/541423/6 "2024-01-25T11:04:14Z")

</div>

Hi @Krzysztof_Jakubczyk

I cloud not answer for a while. I hope you already solve this problem.

Unfortunately, while I debug my code, I initialize the notebook file and I re-coded from the scratch.

In my experience, If you passed `w1_unittest.test_cross_attention(CrossAttention)` there is no problem in self.mha. For me, in the `Decoder` class, I sequentially pass the variable function by function because the given code is based on the functional api.

I don’t remember exactly, as far as I remember, I didn’t pass the variable x.

I hope your successful completion on the Exercise.

Regards,