Course 5 week 4 dot product wrong masked weights

Course Q&A Deep Learning Specialization Sequence Models
1

I did as you mentioned but I still get the following error

I did follow the instruction provided and the TensorFlow library instruction

AssertionError Traceback (most recent call last)
in
30 print("\033[92mAll tests passed")
31
—> 32 scaled_dot_product_attention_test(scaled_dot_product_attention)

in scaled_dot_product_attention_test(target)
23 assert np.allclose(weights, [[0.30719590187072754, 0.5064803957939148, 0.0, 0.18632373213768005],
24 [0.3836517333984375, 0.3836517333984375, 0.0, 0.2326965481042862],
—> 25 [0.3836517333984375, 0.3836517333984375, 0.0, 0.2326965481042862]]), “Wrong masked weights”
26 assert np.allclose(attention, [[0.6928040981292725, 0.18632373213768005],
27 [0.6163482666015625, 0.2326965481042862],

AssertionError: Wrong masked weights

2

Your weight values are incorrect.
There is an error in your code.

3

I knew that, and I can’t figure it out that’s why i wrote this post, could you please help me specify what’s wrong?
i did use the following:
(mask * -1e9)
tf.keras.activations.softmax(scaled_attention_logits, axis=-1)
Appreciate your assistance

4

From the instructions:
image

6 Likes
5

thanks, that’s helpful & solve the issue
may i ask why we changed the value than the recommended one in the library
is it something based on trial & error or based on some analysis that i should know about?

6

The exercise was updated by the instructors.

8

The term (1. - mask) is confusing. You might want to remove the dot after the number 1.

1 Like
9

The dot indicates that the ‘1’ is a floating point value, not an integer.

10

I didn’t get this hint. I don’t think it’s necessary for the 1 to be a float because the term is multiplied by the float -1e9. I’m not criticizing, only giving feedback.

1 Like
11

Thanks for your sugggestion.

12

I have got the same error, and solved by TMosh’s hint.

13

why is it exatly (1.-mask)*-1e9 and not just simply mask*-1e9 ?

14

where are we supposed to use (1-mask)*-1e9 in code? I tried it while adding mask to the scaled_attention_logits, but it is showing me the ‘wrong masked weights’ error. Could you please help me with this issue?

image
image955×248 15.2 KB

15

In the code for scaled_dot_product_attention(), where it has the comment “add the mask to the scaled tensor”.