DLS Course 5 W4 A1 Exercise 3

Course Q&A Deep Learning Specialization Sequence Models
1

Hi,

I cannot figure out what this error is pointing towards. I have printed the “output” “attention_weights” objects. Thnaks for any help.

Carlos.

output = tf.Tensor(
[[0.8112297 0.31122968]
[0.8112297 0.31122968]
[0.7649963 0.3874556 ]], shape=(3, 2), dtype=float32)
attention_weights = tf.Tensor(
[[0.18877034 0.31122968 0.18877034 0.31122968]
[0.18877034 0.18877034 0.31122968 0.31122968]
[0.23500371 0.23500371 0.14253695 0.3874556 ]], shape=(3, 4), dtype=float32)

AssertionError Traceback (most recent call last)
in
1 # UNIT TEST
----> 2 scaled_dot_product_attention_test(scaled_dot_product_attention)

~/work/W4A1/public_tests.py in scaled_dot_product_attention_test(target)
60 assert np.allclose(weights, [[0.2589478, 0.42693272, 0.15705977, 0.15705977],
61 [0.2772748, 0.2772748, 0.2772748, 0.16817567],
—> 62 [0.33620113, 0.33620113, 0.12368149, 0.2039163 ]])
63
64 assert tf.is_tensor(attention), “Output must be a tensor”

AssertionError:

2

Hi @CNP

np.allclose(arg1, arg2) checks arg1 against arg2. If they failed to match, the assertion is triggered.
In this case the assertion is triggered because, the weights returned from the scaled_dot_product_attention_test() failed to match to those weights provided.

There are two places that need checking:

  1. is dk correctly extracted?
  2. adding mask to the scaled tensor - how does you code handle when mask is on?
3

i am doing the square root np.sqrt() of the second dimension of object k (k.shape[1]) for calculating dk

edit: shall I share with you the two lines of code of those two parts or should I message you directly?

Thanks a lot.

Carlos.