C5 W4 A1 scaled_dot_product_attention - Tip

Course Q&A Deep Learning Specialization Sequence Models
1

If you get an error from the tf.keras.activations.softmax() function about “tuple object has no attribute rank”, then be sure you used tf.matmul() when computing Q*K’. and not np.matmul().

The functions do the same thing, but tf.matmul() returns a tensor, which is what the Keras softmax() function requires.

2 Likes
2

Thanks!

I used tf.nn.softmax to generate the tensor, it works too.