C5W4 Programming Ass UNQ_C3

Course Q&A Deep Learning Specialization Sequence Models
1

I’m stuck. Can’t find the problem. I’m applying the mask and -1e9 when computing the scaled_attention_logits. I don’t understand why it is += instead of multiplication. Keep getting the following error.

I’m using tf.nn.softmax to compute attention_weights on the scaled_attention_logits, axis = 1.

AssertionError Traceback (most recent call last)
in
1 # UNIT TEST
----> 2 scaled_dot_product_attention_test(scaled_dot_product_attention)

~/work/W4A1/public_tests.py in scaled_dot_product_attention_test(target)
73 assert np.allclose(weights, [[0.30719590187072754, 0.5064803957939148, 0.0, 0.18632373213768005],
74 [0.3836517333984375, 0.3836517333984375, 0.0, 0.2326965481042862],
—> 75 [0.3836517333984375, 0.3836517333984375, 0.0, 0.2326965481042862]]), “Wrong masked weights”
76 assert np.allclose(attention, [[0.6928040981292725, 0.18632373213768005],
77 [0.6163482666015625, 0.2326965481042862],

AssertionError: Wrong masked weights

2

Did you read this part of the instructions carefully?

image
image804×162 6.18 KB

3

I read, multiple them, so I have the following

(1.-mask) * -1e9

4

Try using tf.keras.activations.softmax(…) instead. You don’t need an “axis” argument.

1 Like
5

wow! so I don’t need an axis, that solved it, thank you!

6

Why is axis a problem here?

7

If you’re going to specify an axis, you need to be sure it’s the correct one.
In this case, the default axis is the correct one.

8

what does this:

Multiply (1. - mask) by -1e9 before applying the softmax.

even mean??