DLS C5 week4 exercise4 - EncoderLayer

Hi all,

I’m having trouble with the EncoderLayer exercise. I am getting an assertion error claiming that “wrong values when training=True”. I’ve looked at some old threads for this issue, and it is getting me nowhere. Furthermore, I’ve tried removing and adding ‘training=training’ into different places to test it.
I wonder if something else is going wrong, but I am being sent back this error instead.

Can anyone help me please?

(I’ve had trouble with the query, key and value. Maybe something is going wrong with that)

Are you still getting an error? If so, please share the screenshot of it.


Yes I am, here is a screenshot of the error:

Thank you.

After reviewing the code, here is my feedback:

First, you don’t need to specify training = training everywhere, other than where mentioned.

Given that:

  1. You will pass the Q, V, K matrices and a boolean mask to a multi-head attention layer. Remember that to compute self -attention Q, V and K should be the same.

So, we have three terms all denoted by x in our code as all are the same here, and then mask.

Also, why do we calculate skip_x_attention ? We need to use that skip_x_attention in self.ffn.