[Week 4] UNQ_C4 "Wrong values when training=True" error

Course Q&A Deep Learning Specialization Sequence Models
1

I’m working through W4’s assignment for the Transformers and am stuck with an error on part UNQ_C4. I’m getting the following error:

---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
<ipython-input-41-00617004b1af> in <module>
      1 # UNIT TEST
----> 2 EncoderLayer_test(EncoderLayer)

~/work/W4A1/public_tests.py in EncoderLayer_test(target)
     92                        [[ 0.23017104, -0.98100424, -0.78707516,  1.5379084 ],
     93                        [-1.2280797 ,  0.76477575, -0.7169283 ,  1.1802323 ],
---> 94                        [ 0.14880152, -0.48318022, -1.1908402 ,  1.5252188 ]]), "Wrong values when training=True"
     95 
     96     encoded = encoder_layer1(q, False, np.array([[1, 1, 0]]))

AssertionError: Wrong values when training=True

Looking at my code, I do have the “training = training” parameter set on the dropout_ffn layer. Does it also need to be set elsewhere? There’s only one dropout layer set up in the code-outline.

Thank you!

2

The only place you use training=training is in dropout_ffn().

A very common error regards which variable you pass to self.ffn() as “the output of the multi-head attention layer”.

Tip: It’s not the variable named attn_output.

2 Likes
3

Thanks for the idea but i’m not passing in that variable … I’m passing the output of the norm of the sum of attn_output + x … (sorry. it’s clearer in code. am I allowed to post that here?)

Actually … found out that I was passing the wrong variables in to the layernorm2 layer … I now get this:

---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
<ipython-input-18-00617004b1af> in <module>
      1 # UNIT TEST
----> 2 EncoderLayer_test(EncoderLayer)

~/work/W4A1/public_tests.py in EncoderLayer_test(target)
     97     assert np.allclose(encoded.numpy(), [[ 0.5167701 , -0.92981905, -0.9731106 ,  1.3861597 ],
     98                            [-1.120878  ,  1.0826552 , -0.8671041 ,  0.905327  ],
---> 99                            [ 0.28154755, -0.3661362 , -1.3330412 ,  1.4176297 ]]), "Wrong values when training=False"
    100     print("\033[92mAll tests passed")
    101 

AssertionError: Wrong values when training=False

So I’m guessing that I still have an error in one of the layers/ what I’m passing in but I don’t see it unfortunately :confused:

4

Nevermind!
I made an error that I found with the way I was calling “training”.
This is solved now.

5

This is very helpful in addtion to the “training=training” advice. I made mistake passing the variables needed for layers. The crosschecking my inputs and figure 2 really helps.

2 Likes
6

That was the key to solving my issue! I was really lost, but rechecking that part made my mistake quite obvious!