C5_W4A1 scaled_dot_product_attention wrong masked values

Course Q&A Deep Learning Specialization Sequence Models
1

AssertionError Traceback (most recent call last)
in
1 # UNIT TEST
----> 2 scaled_dot_product_attention_test(scaled_dot_product_attention)

~/work/W4A1/public_tests.py in scaled_dot_product_attention_test(target)
73 assert np.allclose(weights, [[0.30719590187072754, 0.5064803957939148, 0.0, 0.18632373213768005],
74 [0.3836517333984375, 0.3836517333984375, 0.0, 0.2326965481042862],
—> 75 [0.3836517333984375, 0.3836517333984375, 0.0, 0.2326965481042862]]), “Wrong masked weights”
76 assert np.allclose(attention, [[0.6928040981292725, 0.18632373213768005],
77 [0.6163482666015625, 0.2326965481042862],

AssertionError: Wrong masked weights

any idea where I am doing wrong here it took all day but I am fail to find where I am doing wrong?

2

hi @Sajjad_Ali

Check the below linked comment

regards
DP

2 Likes
3

@Deepti_Prasad is right. Add the mask to the scaled tensor before applying the softmax. The masking operation should look like this:

if mask is not None:

     scaled_tensor += ((1.0 - mask) * -1e9)
1 Like
4

Ah, thanks that was the issue, I just overlooked the instruction and it wasn’t mention in the lecture, thank you for highlighting.

1 Like