C4W2_Assignment 2: Transformer Summarizer InvalidArgumentError

[Deepti_Prasad]

hI @sk_369

1.in scaled dot product grade cell, you don’t require to apply axis when you are calculating attention weights, kindly remove axis=-1

2.your decoder layer codes are incorrect.

2 a. for block 1 codes
Block 1 is a multi-head attention layer with a residual connection, and look-ahead mask. Like in the EncoderLayer, Dropout is defined within the multi-head attention layer.
As the block 1 has 3 multi layer, you require to pass the self.mha1 to 3 linear layer recalled for class DecodeLayer i.e x, x, x with correct mask(which you already used here)

2 b. You are not suppose to apply training to the self attention blocks codes, training is only applied to the dropout ffn_output. Read instructions carefully to avoid such error.

Also refer the below thread for detail instructions for grade cell 2 mistakes

C4W2-Exercise2

Hi @pongyuenlam First of thank you for putting efforts from your side to understanding what you are trying to learn. It feels great as a mentor when we come across learners who also put sincere effort. Once you complete NLP, do complete the Deep Learning Specialisation. Believe me you will not regret. Now comes to your code, I will go step wise. after calculate self attention for block 1(codes were correct for that), you had to apply layer normalization (layernorm1) to the sum of the att…

Do these corrections and let me know in DM, how you correction. Then we will move to next grade cell.

Regards
DP