Optional Lab: Another explanation about PCA

Course Q&A Natural Language Processing NLP with Classification and Vector Spaces
1

Hey Guys,
Can anyone help me out with the “Understanding the transformation model pcaTr” section?

In this section, it is mentioned that “pcaTr.components_ has the rotation matrix”, however, the values stored in pcaTr.components_ ae as follows:

[[-0.70710678 -0.70710678]
[-0.70710678 0.70710678]]

So, shouldn’t the rotation matrix be as follows:

[[-cos(45) -sin(45)]
[-sin(45) cos(45)]]

which is different from the one mentioned in the lab. Also, it is mentioned that “First row must be in the direction of [1, n]”, but for n = 1, isn’t the first row of the rotation matrix in the direction of [-1 -n]. Am I misinterpreting something, if so, please do let me know.

Update #1

In the next section, it is mentioned that “In theory, the Eigenvector matrix must be the inverse of the original rotation matrix”. This also means that the “inverse of the Eigenvector matrix must be the original rotation matrix”. So, even I take the inverse of the above Eigenvector matrix, I should get the rotation matrix as follows:

[[-0.70710678 -0.70710678]
[-0.70710678 0.70710678]]

which is once again not equal to the rotation matrix mentioned in the lab.

Update #2

In Week 4 lab entitled “Rotation matrices in R2”, the rotation matrix is given as

[[-cos(A) -sin(A)]
[-sin(A) cos(A)]]

where A = \theta; I am a little confused as to whether the rotation matrix can have a single possible configuration, or multiple different configurations.

Thanks,
Elemento

2

Hi Elemento,

If you want to rotate a vector r with coordinates (x, y) and angle alpha over an angle theta to get vector r’ with coordinates (x’, y’) then the following holds:

x = r*cos(alpha)
y = r*sin(alpha)

x’ = r’*cos(alpha + theta)
y’ = r’*sin(alpha + theta)

Trigonometric addition algorithms give us:

cos(alpha + theta) = cos(alpha)*cos(theta) - sin(alpha)*sin(theta)
sin(alpha + theta) = cos(alpha)*sin(theta) + sin(alpha)*cos(theta)

As the length of the vector stays the same,

x’ = r*cos(alpha)*cos(theta) - r*sin(alpha)*sin(theta)
y’ = r*cos(alpha)*sin(theta) + r*sin(alpha)*cos(theta)

This equates to

x’ = x*cos(theta) - y*sin(theta)
y’ = x*sin(theta) + y*cos(theta)

Written as matrix multiplication, this becomes

(x’, y’) = (x, y) dot ([[cos(theta), sin(theta)], [-sin(theta), cos(theta)]])

as per the equation of the rotation matrix presented in week 3 lab “Another Explanation about PCA” - but different from the rotation matrix presented in week 4 lab “Rotation matrices in R2”.

The statement that “pcaTr.components_ has the rotation matrix" is contradicted by the sklearn documentation. The documentation indicates that components_ contains the “Principal axes in feature space, representing the directions of maximum variance in the data. Equivalently, the right singular vectors of the centered input data, parallel to its eigenvectors. The components are sorted by decreasing explained_variance_”.

It would be good if this were looked into, which may also help clarifying the issues you mention in your updates.

I will make an issue on github referring to this post. Thanks!

1 Like
3

Hey @reinoudbosch,
Thanks a lot for the detailed explanation.

I also happen to notice that if the vectors were represented in the column representation instead of the row representation, the position of the negative sign will change from the sin in the second row to the sin in the first row.

I guess this could also be mentioned as a side-note, since it is possible for learners to get confused regarding how is the rotation matrix affected by the representation of the vectors.

Cheers,
Elemento

4

Hi Elemento,

Yes, if the rotation matrix is placed before a column vector, the position of the negative sign changes. Also it seems as if changes are made to the rotation matrix to indicate the direction of the rotation, but this would normally be captured by the sign of the angle over which is rotated. This may be confusing to learners.

Great points.

5

Hey @reinoudbosch,
I indeed agree with you. It would be great if a little more explanation regarding the rotation matrix is provided either in the lecture video, for instance, as optional content, or in the assignment itself, as optional understanding.

Cheers,
Elemento