C5_W4_A1_Transformer_Subclass_v1 Unit 3 scaled_dot_product_attention Error

Course Q&A Deep Learning Specialization Sequence Models
1

Greetings,

I tried to utilize what I learned from this link but I still get the following error:

AssertionError Traceback (most recent call last)
in
1 # UNIT TEST
----> 2 scaled_dot_product_attention_test(scaled_dot_product_attention)

~/work/W4A1/public_tests.py in scaled_dot_product_attention_test(target)
60 assert np.allclose(weights, [[0.2589478, 0.42693272, 0.15705977, 0.15705977],
61 [0.2772748, 0.2772748, 0.2772748, 0.16817567],
—> 62 [0.33620113, 0.33620113, 0.12368149, 0.2039163 ]])
63
64 assert tf.is_tensor(attention), “Output must be a tensor”

AssertionError:

This is how I calculate scaled_attention_logits:

{mentor edit: code removed}

I appreciate your help dearly.

2

Your code for matmul_qk is transposing the k matrix twice. That’s a problem.
Either use the .T operator, or use the transpose argument. Not both.

1 Like