Hi Maxime,
The get_angles()
function is to calculate angle, rather than PE (position encoding). Take a look at equations:
Each pair of (even, odd), i.e., (0,1), (2,3), (4,5)…, (2i, 2i+1) for i = 0, 1, 2, …, i, respectively, even and odd have the same angle (red box).
Regarding the limit
(the value at last position and last dimension), the test case generated positions from 0 ~ (position - 1), and dimensions from 0 ~ (d_model -1). So the last value is at (position - 1) and (d_model - 1 = 15). Thus, in the odd equation (pos, 2i+1) = (3, 15). So, i = 7, the angle is: 3/np.power(10000, 2*7/16)
. Or, you can simply take floor of (d_model - 1)//2 to calculate i=7.