Hi,
I am working on the C2W3 Assignment. The notebook said “As regularization is increased, the performance of the model on the training and cross-validation data sets converge. For this data set and model, lambda > 0.01 seems to be a reasonable choice.”
If we pick the lambda say 0.1 which shows that cv error lower than train error, what is the interpretation of the model? will there any negative impact if cv error is lower than train error?
Hello, @ansonchantf,
At any time when dev set error is lower, we can check the size of our dev set. The error is statistical, so if the size is too small, we should not put too much confidence on the value itself. So, would you like to share with us the size of that dev set, just for the sake of continuing the discussion?
Cheers,
Raymond
Hi @ansonchantf ,
If you choose a lambda of 0.1 where the CV error is lower than the training error, it normally indicates that the model is not overfitting and generalizes well to new unseen data. However, if the CV error is significantly lower than the training error, it shows that the training data has noise or outliers (anomalies) causing higher error. Ideally, you want the training and CV errors to be close, indicating a good balance between bias and variance. Choosing a lambda that minimizes both errors without a large difference is the best practice.
Hope this helps!
