Your positional encoding and add to word embedding is correct but
< # scale embeddings by multiplying by the square root of their dimension needs a look up,
looks like you have missed to scale your embedding by multiplying it by the square root of your embedding dimension. You only casted the embedding dimension to data type tf.float32 before computing the square root.
So make sure you are using the correct tf function call for that code line.
Incase you are not able to find, you can send me the notebook via personal DM.
The auto-grader gave an error which I think might be related
Code Cell UNQ_C2: Unexpected error (ValueError('operands could not be broadcast together with shapes (1,4,8) (1,16,8) ')) occurred during function check. We expected function positional_encoding to return type <class ‘tuple’>. Please check that this function is defined properly.
Casper can you please send me the notebook via personal DM. Click my name and then message.
you have tuple the self_embedding_dim and tf.float32 incorrectly. You also have not recalled that code line with one tf. function which causing positional coding error.