In the Optional Lab for Model Evaluation and Selection, in the last part where NNs are applied for Classification, it is suggested that if the CV error is the same for 2 models (in this case NNs 2 and 3 in the workbook), we should choose the model with the lower training error, which makes sense.
But in this particular case, NN 2 is simpler than NN 3 (5 layers vs 6 layers), both give same CV error and NN 3 obviously has lower training error, being the more complex NN. So, both NN 2 and NN 3 fit the CV set equally well, but NN 2 is simpler.
So could a case also be made for choosing NN 2 rather than NN 3 because it’s simpler? Like in a real-world scenario, if 2 NNs give the same CV error, but one is far simpler than the other, wouldn’t it be preferable to choose the simpler one? (considering computational resources required etc)