C5W4A1 Exercise 6 "Wrong values in out"

Hello everyone. I looked around for posts with similar issues but couldn’t find any. So any ideas or things for troubleshoot for is appreciated.

So far I have checked:

  1. All of the residual connections

  2. input of mha2

  3. training=training in the drop out later
    Although I am not sure if that’s necessary

Thank you!

AssertionError                            Traceback (most recent call last)
<ipython-input-199-7c54c129d8b0> in <module>
      1 # UNIT TEST
----> 2 DecoderLayer_test(DecoderLayer, create_look_ahead_mask)

~/work/W4A1/public_tests.py in DecoderLayer_test(target, create_look_ahead_mask)
    180     assert np.allclose(attn_w_b1[0, 0, 1], [0.5271505,  0.47284946, 0.], atol=1e-2), "Wrong values in attn_w_b1. Check the call to self.mha1"
    181     assert np.allclose(attn_w_b2[0, 0, 1], [0.32048798, 0.390301, 0.28921106]),  "Wrong values in attn_w_b2. Check the call to self.mha2"
--> 182     assert np.allclose(out[0, 0], [-0.22109576, -1.5455486, 0.852692, 0.9139523]), "Wrong values in out"

AssertionError: Wrong values in out

“training = training” is used in Decoder(), but not in DecoderLayer().

Thank you for the prompt response. However, even when I remove the “training = training”, I still get the same error

For reference, my out3 looks something like this

[[[-0.16047186 -1.5689311   0.9576684   0.77173454]
  [-1.6884431   0.47859752  0.91171324  0.2981322 ]
  [ 0.72350425 -0.97536105 -0.99065024  1.242507  ]]], shape=(1, 3, 4), dtype=float32)

Here are mine for that unit test:

Check how you used mha1() and mha2().

I used

for mha1()

for mha2()

And they seem right to me. I went through my code again to double check for subtle misspells and didn’t find any

I’ll clean up all of the code that appears in this thread.

For others who find this thread later: Check that you’re using the correct data in the dropout_ffn(…) layer.

Thank you! Problem solved

I also had this error. In my case I’d made a mistake in layernorm2. hth.