Wrong values EncoderLayer

Why getting wrong values in EncoderLayer?
how many arguments should we give to self.mha?
should mask be modified or directly be given as an argument to self.mha?
how about x?should it be modified or sliced?


(x,x,x,mask) are the parameters.

1 Like

Hi, I ve got stuck with wrong encoder layer values in test: not sure, if I am getting the syntax of tensorflow right, even when following the instructions:
i.e. 2. Next, you will pass the output of the multi-head attention layer to a dropout layer. Don’t forget to use the training parameter to set the mode of your model. β†’ like this: self.dropout1(self_attn_output, training)?
or 3. Now add a skip connection by adding your original input x and the output of the dropout layer. β†’ self.layernorm1(self_attn_output+x)?

Hi. Did you remember to use the mask? it must be passed to the MultiHeadAttention layer.