Positional_encoding function

Course Q&A Deep Learning Specialization Sequence Models
1

I have a particular question regarding these formulas,

image
why is their implementation is :
angle_rads[:, 0::2] = np.sin(angle_rads[:, 0::2])
angle_rads[:, 1::2] = np.cos(angle_rads[:, 1::2])
instead of :
angle_rads[:, 0::2] = np.sin(angle_rads[:, 0::2])
angle_rads[:, 1::2] = np.cos(angle_rads[:, 0::2])
what is inside the sin and cos parentheses is similar.

2

Hey @Sofiane,

The reason is that the angle_rads matrix has been initialized with all possible values for the sine and cosine functions. Each column provides values for a curve. The nearest columns (e.g. 1th and 2nd, 3rd and 4th, etc.) have already been initialized with the same values.

You can find a bit more on position encoding in this post.