C5W4 Programming Ass UNQ_C3

I’m stuck. Can’t find the problem. I’m applying the mask and -1e9 when computing the scaled_attention_logits. I don’t understand why it is += instead of multiplication. Keep getting the following error.

I’m using tf.nn.softmax to compute attention_weights on the scaled_attention_logits, axis = 1.

AssertionError Traceback (most recent call last)
----> 2 scaled_dot_product_attention_test(scaled_dot_product_attention)

~/work/W4A1/public_tests.py in scaled_dot_product_attention_test(target)
73 assert np.allclose(weights, [[0.30719590187072754, 0.5064803957939148, 0.0, 0.18632373213768005],
74 [0.3836517333984375, 0.3836517333984375, 0.0, 0.2326965481042862],
—> 75 [0.3836517333984375, 0.3836517333984375, 0.0, 0.2326965481042862]]), “Wrong masked weights”
76 assert np.allclose(attention, [[0.6928040981292725, 0.18632373213768005],
77 [0.6163482666015625, 0.2326965481042862],

AssertionError: Wrong masked weights

Did you read this part of the instructions carefully?

I read, multiple them, so I have the following

(1.-mask) * -1e9

Try using tf.keras.activations.softmax(…) instead. You don’t need an “axis” argument.

1 Like

wow! so I don’t need an axis, that solved it, thank you!

Why is axis a problem here?

If you’re going to specify an axis, you need to be sure it’s the correct one.
In this case, the default axis is the correct one.

what does this:

Multiply (1. - mask) by -1e9 before applying the softmax.

even mean??