Do you see regularization the way I do?

Course Q&A Machine Learning Specialization Supervised ML: Regression and Classification
1

I assume that we cannot know the exact form of our hypothesis that would fit the training examples.

That’s why we initially give the hypothesis more freedom by making it non-linear and adding more parameters. That way we run away from the problem of underfitting.
However, this makes our model more vulnerable to overfitting :slight_smile:. Overfitting would occur especially if we do not have a sufficient amount of training examples.

We fight overfitting by regularization - a technique that prevents giving a single parameter too much weight. Instead it distributes the weight more evenly. At least that is my assumption. What do you think?

The critical part here would be to determine the correct value for lambda. I would personally run the cost function for different values of lambda and then take the one that has the smallest global minimum. I am looking forward to understand what Andrew would have to say on that in course 2.

2

Thanks for your post.

The purpose of regularization is to reduce the model complexity by penalising it and so reduce overfitting. So it’s about reducing the model dependency of many parameters, e.g. by:

  • driving weights exactly to zero (L1 regularization) or
  • driving weights close to zero (L2 regularization)

Also dropout is a useful technique to tackle overfitting.

I like these explanation here, too. Feel free to take a look:
-Regularization for Simplicity: L₂ Regularization  |  Machine Learning  |  Google for Developers

Best regards
Christian

1 Like
3

Thanks for the link. It was very useful!

1 Like