May I know why we do not need the orange part in the formula when do the testing for the model?
Regularization (the orange part) is only used in training. It creates an extra penalty for having large weight values.
Once you have a model, you don’t want to include those extra penalties - you just want to compute how well the model fits the test set.
1 Like
Hello @Panda_Food,
Let’s think about how we are going to use it -
-
if we are to compare the performance of two models, then only the prediction errors matter and that is already provided by the yellow part. The orange part is totally irrelevant to the errors but it may change the comparison result undesirably - e.g. Model 2 has less prediction error and should thus be a better model but, due to the increased orange part, appears to be worse.
-
if we are to compare different set of data against the same model which means the weights are all constant throughout those comparisons, then the orange part is just a constant and thus meaningless!
Cheers,
Raymond
1 Like