DLS C5 week4 exercise4 - EncoderLayer

Course Q&A Deep Learning Specialization Sequence Models
1

Hi all,

I’m having trouble with the EncoderLayer exercise. I am getting an assertion error claiming that “wrong values when training=True”. I’ve looked at some old threads for this issue, and it is getting me nowhere. Furthermore, I’ve tried removing and adding ‘training=training’ into different places to test it.
I wonder if something else is going wrong, but I am being sent back this error instead.

Can anyone help me please?

2

(I’ve had trouble with the query, key and value. Maybe something is going wrong with that)

3

Are you still getting an error? If so, please share the screenshot of it.

4

Hi,

Yes I am, here is a screenshot of the error:

image
image1056×441 24.5 KB

Thank you.

5

After reviewing the code, here is my feedback:

First, you don’t need to specify training = training everywhere, other than where mentioned.

Given that:

  1. You will pass the Q, V, K matrices and a boolean mask to a multi-head attention layer. Remember that to compute self -attention Q, V and K should be the same.

So, we have three terms all denoted by x in our code as all are the same here, and then mask.

Also, why do we calculate skip_x_attention ? We need to use that skip_x_attention in self.ffn.