C5W4A1 Exercise 6 "Wrong values in out"

Course Q&A Deep Learning Specialization Sequence Models
1

Hello everyone. I looked around for posts with similar issues but couldn’t find any. So any ideas or things for troubleshoot for is appreciated.

So far I have checked:

  1. All of the residual connections

  2. input of mha2

  3. training=training in the drop out later
    Although I am not sure if that’s necessary

Thank you!

---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
<ipython-input-199-7c54c129d8b0> in <module>
      1 # UNIT TEST
----> 2 DecoderLayer_test(DecoderLayer, create_look_ahead_mask)

~/work/W4A1/public_tests.py in DecoderLayer_test(target, create_look_ahead_mask)
    180     assert np.allclose(attn_w_b1[0, 0, 1], [0.5271505,  0.47284946, 0.], atol=1e-2), "Wrong values in attn_w_b1. Check the call to self.mha1"
    181     assert np.allclose(attn_w_b2[0, 0, 1], [0.32048798, 0.390301, 0.28921106]),  "Wrong values in attn_w_b2. Check the call to self.mha2"
--> 182     assert np.allclose(out[0, 0], [-0.22109576, -1.5455486, 0.852692, 0.9139523]), "Wrong values in out"
    183 
    184 

AssertionError: Wrong values in out
2

“training = training” is used in Decoder(), but not in DecoderLayer().

3

Thank you for the prompt response. However, even when I remove the “training = training”, I still get the same error

4

For reference, my out3 looks something like this

tf.Tensor(
[[[-0.16047186 -1.5689311   0.9576684   0.77173454]
  [-1.6884431   0.47859752  0.91171324  0.2981322 ]
  [ 0.72350425 -0.97536105 -0.99065024  1.242507  ]]], shape=(1, 3, 4), dtype=float32)
5

Here are mine for that unit test:

image
image786×220 9.79 KB

6

Check how you used mha1() and mha2().

7

I used

for mha1()

for mha2()

And they seem right to me. I went through my code again to double check for subtle misspells and didn’t find any

9

I’ll clean up all of the code that appears in this thread.

For others who find this thread later: Check that you’re using the correct data in the dropout_ffn(…) layer.

10

Thank you! Problem solved

11

I also had this error. In my case I’d made a mistake in layernorm2. hth.