Hi,
I am getting the error saying ‘AssertionError: Wrong unmasked weights’. I have scaled matmul_qk. I have also added the mask to scaled_attention_mask. I have also added the term (1. - mask) * -1e9. But i am skeptical about the correct position where i should be adding the (1. - mask)* -1e9 term. Can anyone please help me with this issue?
I think this post is a duplicate. I answered a similar question from you just a minute ago on a different thread.