Perhaps I’m mistaken but doesn’t the formula
Create a divide by zero Nan output for i=0?
Nvm. I was reading the formula wrong. The 2i/d is an exponent
Thank you for sharing the solution to your query!
Ahhhh thank you, I’ve been fighting NaNs and INFs for an hour and was wondering what on earth was going on.
As a bit of feedback to the course organisers, would it be possible to have an optional video that builds up the positional encodings a bit more from first principles?
The explanation provided by Andrew provides a reasonable intuition, but I would have liked to build up a better understanding of the fundamental mathematics / Trigonometry at play.