Scales dot product attention

[isuru]

in scaled_dot_product_attention_test(target)
10 assert np.allclose(weights, [[0.2589478, 0.42693272, 0.15705977, 0.15705977],
11 [0.2772748, 0.2772748, 0.2772748, 0.16817567],
—> 12 [0.33620113, 0.33620113, 0.12368149, 0.2039163 ]])

I got the error like this please help me I am stuck at this place for 4 hours now
if mask is not None:
scaled_attention_logits += mask * -1e9
is the above statement is write
please help

I used-> dk = k.shape[-1]
scaled_attention_logits = matmul_qk/np.sqrt(dk)

also
used tf.keras.activations.softmax(scaled_attention_logits, axis = dk) to get attention weights

[MichaelAnderson]

Hi @isuru use tf.linalg.matmul for the output, that should fix it

[TMosh]

@isuru
I do not see any issues there.

Please edit your message and remove your code fragments. Posting your code breaks the Honor Code.