AssertionError Traceback (most recent call last)
in
1 # UNIT TEST
----> 2 scaled_dot_product_attention_test(scaled_dot_product_attention)
~/work/W4A1/public_tests.py in scaled_dot_product_attention_test(target)
73 assert np.allclose(weights, [[0.30719590187072754, 0.5064803957939148, 0.0, 0.18632373213768005],
74 [0.3836517333984375, 0.3836517333984375, 0.0, 0.2326965481042862],
—> 75 [0.3836517333984375, 0.3836517333984375, 0.0, 0.2326965481042862]]), “Wrong masked weights”
76 assert np.allclose(attention, [[0.6928040981292725, 0.18632373213768005],
77 [0.6163482666015625, 0.2326965481042862],
AssertionError: Wrong masked weights
any idea where I am doing wrong here it took all day but I am fail to find where I am doing wrong?
hi @Sajjad_Ali
Check the below linked comment
regards
DP
2 Likes
@Deepti_Prasad is right. Add the mask to the scaled tensor before applying the softmax. The masking operation should look like this:
if mask is not None:
scaled_tensor += ((1.0 - mask) * -1e9)
1 Like
Ah, thanks that was the issue, I just overlooked the instruction and it wasn’t mention in the lecture, thank you for highlighting.
1 Like