C5_W4_A1_Transformer_Subclass_v1, UNQ_C4, EncoderLayer error: "Wrong values when training=True"

Course Q&A Deep Learning Specialization Sequence Models
1

Hi
I am getting error with UNQ_C4 “EncoderLayer” but I am unable to understand what is the issue from the error message. Can someone please help me understand what is the issue with my EncoderLayer definition?

Screen Shot 2023-04-20 at 9.20.03 PM
Screen Shot 2023-04-20 at 9.20.03 PM1952×596 76.6 KB

1 Like
2

Please click my name and message your notebook as an attachment. Send the notebook in ipynb format and not in pdf / image format.

1 Like
3

Sent you my notebook ipynb. Thanks for taking a look

4

You’re welcome.

There are 2 things to remember:

  1. You should pass training flag to the layers where necessary.
  2. The encoder_layer_out calculation is incorrect. See this: output from multi-head attention (skip connection). It means you should take the output of # skip connection.
5

Thanks Balaji
Your comment help me identify and fix the problem in UNQ_C4
I’ve been stuck with a similar problem in UNQ_C6 that I can;t understand now. Can you please help me understand what is the problem here?

Screen Shot 2023-04-21 at 3.26.57 PM
Screen Shot 2023-04-21 at 3.26.57 PM1980×650 98.1 KB

6

Please click my name and message your notebook as an attachment. Send the notebook in ipynb format and not in pdf / image format.

7

Sent you ipymb attachment, thanks

8

Please fix the argument when calling self.layernorm2. See from the decoder layer diagram that output of the 1st batch norm layer is taken towards the 2nd Add & Norm block.

1 Like
9

Thanks for the help Balaji, your comment helped me fix the issue

1 Like