C5_W4_A6 Decoder Layer

Course Q&A Deep Learning Specialization Sequence Models
1

Hi,
I have a problem with the Decoder Layer, I got the Wrong values in out and I don’t know why…

---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
<ipython-input-205-7c54c129d8b0> in <module>
      1 # UNIT TEST
----> 2 DecoderLayer_test(DecoderLayer, create_look_ahead_mask)

~/work/W4A1/public_tests.py in DecoderLayer_test(target, create_look_ahead_mask)
    180     assert np.allclose(attn_w_b1[0, 0, 1], [0.5271505,  0.47284946, 0.], atol=1e-2), "Wrong values in attn_w_b1. Check the call to self.mha1"
    181     assert np.allclose(attn_w_b2[0, 0, 1], [0.32048798, 0.390301, 0.28921106]),  "Wrong values in attn_w_b2. Check the call to self.mha2"
--> 182     assert np.allclose(out[0, 0], [-0.22109576, -1.5455486, 0.852692, 0.9139523]), "Wrong values in out"
    183 
    184 

AssertionError: Wrong values in out

I checked the other topics but it doesn’t seem that I have the same problems. I checked that I am using the correct layer names, the correct order for Q,K and V (cf Course 5: Week 4 Transformer DecoderLayers - #14 by TMosh. So right now I really don’t know what to do to solve the unit test. Have any idea ?

Thanks

3 Likes
2

The thread you mentioned is 10 days old. The Notebook for this exercise was updated two days ago. So that 10-day-old thread my not be accurate.

3

Hi, I know but it still not working for me, can you help me getting what’s wrong with my code ?

mentor edit: code removed

Thanks a lot !

4

Your layernorm2() layer is using an incorrect data argument.
The “output of the first block” is Q1.

3 Likes
5

This post really saved my life!!! Thanks a lot.

1 Like
6

I wonder how you tell that the output of the first block is Q1?
Does the flowchart show it? Where can I find some clues?

7

You can tell because “Q1 = …” is the last line of code in the section that is commented “Block 1”.

2 Likes
8

i did everything as instructed still i am getting this error:
“Wrong values in attn_w_b1. Check the call to self.mha1”
Any help would be appreciated. Thanks!

1 Like
9

I understand now. Thank you.

10

Hi, thanks a lot, it was that !

11

I’m getting the same error (wrong value in out) even though the values I am passing to the layernorm2() are correct (mult_attn_out2 and Q1).

12

NVM - I was missing one argument in the 3rd normalization layer. Issue is fixed