I am struggling with the compute_pca function. I have rewatched the videos for help, but it still is not clear how I do the final calculations in the compute_pca function. I’ve summarized the steps I think I have correct.
I start by mean centering X. This still has dimensions 10,3
I then get the covariance_matrix by passing the X_demeaned to the numpy covariance function. This gives me a 3x3 matrix.
I pass this to the eigh function which returns eigen_vals (3,) and eigen_vecs (3,3).
I then get an index list for sorted eigen_vals. This is in ascending order and so I flip the order to descending.
I then apply this ordering to the eigen_vals and this looks correct. The ones with most significance are at the top now (confirmed visually).
I then narrow this to the top n_components giving me a subset (3,2).
All good up to this point I think. What I am struggling with, given that I don’t have a ton of linear algebra experience is how to get X_reduced in the final step.
The instructions say to transform the data by multiplying the transpose of the eigenvectors with the transpose of the de-meaned data and then take the transpose of that product.
The transpose of the eigenvectors subset is (3,2) and the transpose of the de-meaned data is (10,3). I tried taking the dot product of the de-meaned data and the eigenvectors subset and then transposing that. The only way I could get it to work was to put the demeaned data first and then the subset which gives me a (10,2) result. The transpose is (2,10) which doesn’t look right. I’ve tried to find an example of the necessary calculation in the lecture material but I am not seeing it. It little more explanation on this would help greatly.
So you can see that the X_reduced shape should be 3 x 2, not 2 x 10.
One other thing to note is that perhaps their instructions are a bit more complex than they really need to be. Note the following mathematical relationship:
(A \cdot B)^T = A^T \cdot B^T
But maybe the more relevant way to use that mathematical fact is to write it this way:
(A^T \cdot B^T)^T = (B^T)^T \cdot (A^T)^T = B \cdot A
Thank you for the print statements. I appear to be doing something wrong with the covariance call. I am using the np.cov() function passing in X_demeaned as only argument. Do I need additional arguments? Am I using the wrong call?
Here are my corresponding print statements up through covariance.
X.shape (3, 10)
X_demeaned.shape (3, 10)
covariance_matrix.shape (3, 3)
You have two choices in how you invoke np.cov to get the right answer. If you pass X_demeaned as the argument, then you need to also pass rowvar = False to get the correct answer.
Or you can pass X_demeaned.T and use rowvar = True.
Thanks for the print statements. Adding the argument to the cov function fixed the issue with the dimensions for the covariance matrix, but my eigen_vals don’t match yours. That makes me wonder if I am not calculating the X_demeaned values correctly. I am currently subtracting the mean of X from each of the elements. Was this the correct approach. My output up to calculating X_reduced is pasted below: