Hey everybody
In the assignment, the function get_angles(...)
initializes a matrix of all possible values for the sine and cosine functions that we use later to calculate positional encoding.
In this video Andrew explains that a sine curve matches the following cosine curve, that means the values for both curves should be the same. In out matrix initialized by get_angles(...)
values for each curve are stored in columns, and i
specifies a column index for each curve. To make the values same between neighboring columns we use i // 2
in our implementation.