Hello,
In the Advanced Learning Algorithms course, the video titled Deciding what to try next, the slide titled Debugging a learning algorithm: If I used all the features of the data set, shouldn’t the regularization term drive down the parameters for those features that do not contribute meaningfully to the prediction?
Thank you.
Questions like this always have some subtleties to them, but my interpretation would be different. Note that the L2 regularization term involves the square of the w_j values. When you square a number less than one, it gets smaller, right? And the smaller it is to start with, the more the reduction in value you get from the squaring, of course. So my interpretation would be that the L2 regularization will have a bigger effect on the features that the base training (driven by the original error term) has given higher weight values (more influence on the outcome).
But that’s just the first lecture in this week, so my suggestion would be to “hold that thought” and listen to all that Professor Ng says about how to make the decisions when your model isn’t giving you the performance that you want.