This is the error I keep getting and I have no idea what this means.
AttributeError Traceback (most recent call last)
in
1 # UNIT TEST
----> 2 scaled_dot_product_attention_test(scaled_dot_product_attention)
~/work/W4A1/public_tests.py in scaled_dot_product_attention_test(target)
55 v = np.array([[0, 0], [1, 0], [1, 0], [1, 1]]).astype(np.float32)
56
—> 57 attention, weights = target(q, k, v, None)
58 assert tf.is_tensor(weights), “Weights must be a tensor”
59 assert tuple(tf.shape(weights).numpy()) == (q.shape[0], k.shape[1]), f"Wrong shape. We expected ({q.shape[0]}, {k.shape[1]})"
in scaled_dot_product_attention(q, k, v, mask)
35 # softmax is normalized on the last axis (seq_len_k) so that the scores
36 # add up to 1.
—> 37 attention_weights = tf.keras.activations.softmax(scaled_attention_logits,axis=-1) # (…, seq_len_q, seq_len_k)
38
39 output = np.matmul(attention_weights,v) # (…, seq_len_q, depth_v)
/opt/conda/lib/python3.7/site-packages/tensorflow/python/util/dispatch.py in wrapper(*args, **kwargs)
199 “”“Call target, and fall back on dispatchers if there is a TypeError.”“”
200 try:
→ 201 return target(*args, **kwargs)
202 except (TypeError, ValueError):
203 # Note: convert_to_eager_tensor currently raises a ValueError, not a
/opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/activations.py in softmax(x, axis)
72 ValueError: In case dim(x) == 1.
73 “”"
—> 74 rank = x.shape.rank
75 if rank == 2:
76 output = nn.softmax(x)
AttributeError: ‘tuple’ object has no attribute ‘rank’
I’ve looked through the other threads and everything I’ve done looks correct to me (checked that it’s matmul with k transpose, dk is k.shape[-2], mask is added with *, last step done with matmul again) but I still keep getting this. Any help would be appreciated!