C5_W4A1 scaled_dot_product_attention wrong masked values

[Sajjad_Ali]

AssertionError Traceback (most recent call last)
in
1 # UNIT TEST
----> 2 scaled_dot_product_attention_test(scaled_dot_product_attention)

~/work/W4A1/public_tests.py in scaled_dot_product_attention_test(target)
73 assert np.allclose(weights, [[0.30719590187072754, 0.5064803957939148, 0.0, 0.18632373213768005],
74 [0.3836517333984375, 0.3836517333984375, 0.0, 0.2326965481042862],
—> 75 [0.3836517333984375, 0.3836517333984375, 0.0, 0.2326965481042862]]), “Wrong masked weights”
76 assert np.allclose(attention, [[0.6928040981292725, 0.18632373213768005],
77 [0.6163482666015625, 0.2326965481042862],

AssertionError: Wrong masked weights

any idea where I am doing wrong here it took all day but I am fail to find where I am doing wrong?

[Deepti_Prasad]

hi @Sajjad_Ali

Check the below linked comment

Help needed for C5W4A1 EX-3

Your error is stating to check codes for masked weight i.e. add the mask to the scaled tensor. The boolean mask parameter can be passed in as none or as either padding or look-ahead. Multiply ((1. - mask) * -1e9) before applying the softmax. Perhaps check if you missed that extra tuple as you add to the scaled attention logits, not placing the tuple can cause error. Regards DP

regards
DP

[nadtriana]

@Deepti_Prasad is right. Add the mask to the scaled tensor before applying the softmax. The masking operation should look like this:

if mask is not None:

     scaled_tensor += ((1.0 - mask) * -1e9)

[Sajjad_Ali]

Ah, thanks that was the issue, I just overlooked the instruction and it wasn’t mention in the lecture, thank you for highlighting.