What is the relation between Lambda(Regularization Parameter) and Weight? Is there any formula between them?
Hi, @Deependra.
Lambda is a hyperparameter that multiplies the regularization term, which is a function of the weights, to adjust its impact. More precisely, increasing lambda strengthens the effect of regularization.
You have all the formulas in this lecture 
Hello @nramon,
I’ve just rewatched the lecture. I still didn’t see how exactly lambda is related to the weights. Professor Ng stated that increasing the value of lambda makes the weights smaller so I assume they are inversely related. Is there an explicit formula that shows how the two are related? Or is the regularization term fixed to a certain value such that when you increase the value of lambda, the weights get smaller?
Thank you so much in advance. Looking forward to hearing from you.
Regards,
Nay
Hi, @Nay.
This is the expression that relates them:
But remember that you’re trying to minimize J.That is, find the values w^{[1]}, b^{[1]}, ..., w^{[L]}, b^{[L]} that make it as small as possible. All else being equal, increasing \lambda makes the penalty term larger, so once \lambda is fixed, how can you make this term smaller?