I’m having big trouble with Week2 MOOC 4 Transformer Summarizer Assignment, since i’ve got all tests passed but I got no points graded, can you help me solve this problem
You probably made the same mistake.
Note, that when creating padding mask for the decoder’s second attention block - we use the encoder_input. In other words, we inform the decoder to not pay attention to padding tokens of the document to be summarized.
Also note, that this is different from look_ahead_mask (causal mask) where decoder is only allowed to pay attention to itself and its previous tokens.
Cheers
Hi @arvyzukai
I am also facing the same issue, but as you mentioned that we need to use encoder_input but within the instructions it clearly mentions that we need to use encoder_output or am I missing something?
Attached the screenshot of the same. Refer point no. 2
Hi @Karan_Bari
These are two different exercises. The one you did attach the screenshot is from “Exercise 3 - Decoder”, the one that I linked in the above post is from “Exercise 5 - next_word”. So maybe you’re not facing the same issue.
If I’m not mistaking, according to community guidelines you should open a new topic when your issue is not the same as in the Original post.
But just to quick comment on the Exercise 2 - we do use encoder output in the Block 2, as we also use the Block’s 1 output. If it still confuses you, you should open another thread.
Cheers