Advice needed for neural network with regularization

[francktchafa]

• List item

I have a few questions, please:

1. Is regularization required for neural networks?
2. When applying regularization to neural networks, do we need to optimize lambda? If yes, do we look at the training and cross-validation error for various lambda values and then choose the one corresponding to the lowest cross-validation error?

[TMosh]

Regularization is never “required”, but it often is helpful in avoiding overfitting the training set.

Yes, the lambda value must be optimized. Otherwise, you have no idea what value is best.

The optimization is the same method as for other regressions - use the results on a validation set to adjust the lambda used in training.

[francktchafa]

Thanks for the prompt reply!

[Erfan_Ram]

i trained my model first without applying Regularization term ( right side ). and then applying Regularization the way Andrew applied to the model( left side ).You can see the differences and how it might affect preventing overfitting.