Exercise 3 - scaled_dot_product_attention AssertionError

Course Q&A Deep Learning Specialization Sequence Models
1

AssertionError Traceback (most recent call last)
in
1 # UNIT TEST
----> 2 scaled_dot_product_attention_test(scaled_dot_product_attention)

~/work/W4A1/public_tests.py in scaled_dot_product_attention_test(target)
60 assert np.allclose(weights, [[0.2589478, 0.42693272, 0.15705977, 0.15705977],
61 [0.2772748, 0.2772748, 0.2772748, 0.16817567],
—> 62 [0.33620113, 0.33620113, 0.12368149, 0.2039163 ]])
63
64 assert tf.is_tensor(attention), “Output must be a tensor”

AssertionError:

I don’t know where I am getting wrong:
I used (dk, col) = np.shape(k)
and checked that (1. - mask)*-1e9
then layerNormalization
then at output : tf.matmul with v

Please Help!

2 Likes
2

You should not use layer normalization in scaled dot product attention

1 Like
3

Yeah! Thank You, I was just completely confused with this line : softmax is normalized on the last axis (seq_len_k) so that the scores add up to 1.
Now I got it, All test passed
Thank You.