C3-week2-PCA, Principal component analysis vs regularization

Course Q&A Machine Learning Specialization Unsupervised Learning, Recommenders, Reinforcement
1

Hello -
Regarding class 3, week 2, on the topic of PCA. I understand the differences in how regularization differs from PCA in the calculations/analysis/approaches. However, in my mind, I guess I always kind of thought of them as having the same use, or, end goal of: “gives the modeler the freedom to worry less about deciding which features matter in the model.” – maybe I was wrong in that thought…

I think of PCA this way: it can be used at the start of analysis to reduce the number of features, then the model can be trained/tested on the new components, which makes me worry less about deciding which features matter in the model.

And I think of regularization this way: it ignores the features that don’t have a large impact, which makes me worry less about deciding which features matter in the model.

Two questions here:
1.) In the course, reducing the features for use in training/testing of a model was either not mentioned or I missed it, and visualization was the main use for PCA. Am I wrong in the idea that PCA can be used to reduce features before training/testing, to help us worry less about which features are being used?

2.) Why would anybody use PCA (besides visualization) instead of regularization? It seems like regularizing during training/testing solves the “too many features” problem pretty well.

Thanks.

2

PCA is primarily use to reduce the dimensionality of a data set - primary down to 2D or 3D so it can be visualilzed.

It isn’t inherently a regularization tool. Although that is a side-effect of throwing away a lot of features.

Regulariation is how you avoid overfitting the training set.

PCA isn’t much used anymore. Used to be a Big Deal to use PCA before you did any training - but that was primarily due to ancient computers that had minimal memory and processor speed.

2 Likes