C5 W4 A1: AssertionError: Wrong masked weights

Hi, I have tried all the solutions applied in the two threads link1 and link2 but none worked.
to be specific I did following:

  1. applied -1e9 to the mask before adding up to scaled tensor.
  2. used tf.keras.activations.softmax(scaled_attention_logits, axis=-1)
  3. used tf.cast(tf.shape(k)[-1], tf.float32) to get dk
  4. used tf.matmul(q, k, transpose_b=True)
    but I am still getting error: AssertionError: Wrong masked weights

I also refreshed my notebook but same issue happened again.
Since I can’t post my full code so I am posting parts only. ( I can send my code in message if you want to have a look at it)

Reminder : … . Multiply (1. - mask) by -1e9 before applying the softmax.
Adding the wrong quantity to the scaled_attention_logits might be the problem.


Thanks @Ihme11 ,
This was the problems.

@Ihme11 - Hi I have just hit this same issue.

I thought I had already applied this step in the below snippet:

# add the mask to the scaled tensor.
    if mask is not None:
        scaled_attention_logits += (1. - mask * -1.0e9)

Is this in the wrong place?

1 Like

operator precedence …


Ahaha lost about 1h30 because of overlooking operator precedence, thanks ans

i have same issues how to over come it?