[Week 4] Exercise 1 - get_angles

Hi Maxime,

The get_angles() function is to calculate angle, rather than PE (position encoding). Take a look at equations:
image
Each pair of (even, odd), i.e., (0,1), (2,3), (4,5)…, (2i, 2i+1) for i = 0, 1, 2, …, i, respectively, even and odd have the same angle (red box).

Regarding the limit (the value at last position and last dimension), the test case generated positions from 0 ~ (position - 1), and dimensions from 0 ~ (d_model -1). So the last value is at (position - 1) and (d_model - 1 = 15). Thus, in the odd equation (pos, 2i+1) = (3, 15). So, i = 7, the angle is: 3/np.power(10000, 2*7/16). Or, you can simply take floor of (d_model - 1)//2 to calculate i=7.

2 Likes