@Fedi_ZOUARI, are you looping through all the required layers? Also worth checking, the params passed to self.pos_encoding and self.dropout. I recall having had some struggles there.

I have not gotten to this section of Course 5 yet, so I can’t help you with the logic here. Better to listen to what Juan says on that. But there is an important general point to make: an error in the third decimal place is not a rounding error. In 64 bit floating point, rounding errors are typically of order 10^{-16} or smaller in the mantissa. Even in 32 bit floating point, the resolution of the mantissa is slightly better than 10^{-7}. So an error in the third decimal place represents some kind of real error in your code.

Thanks all for your replies… I understood that I should not have called the error as “rounding error”. I also read the notebook again with all hints and details, however, I am still unable to find the mistake that I am making! Could anyone give me some hints on how I could find out my error or could someone check my notebook? Thanks in advance!

@Fedi_ZOUARI I will be more than happy to check your code and provide a more focused hint. Please send me a direct message with your code and this cue:

Thanks to @TMosh and @Juan_Olano I was able to solve it. I had an error in the scaling of the embedding where I multiplied by the square root of the sequence length.