[week 4] Transformer Network - get_angles

Hey everybody :slightly_smiling_face:

In the assignment, the function get_angles(...) initializes a matrix of all possible values for the sine and cosine functions that we use later to calculate positional encoding.

In this video Andrew explains that a sine curve matches the following cosine curve, that means the values for both curves should be the same. In out matrix initialized by get_angles(...) values for each curve are stored in columns, and i specifies a column index for each curve. To make the values same between neighboring columns we use i // 2 in our implementation.

@andrew1 @rogeriovazp

8 Likes