Sequel to my previous question here How to evaluate accuracy of a regression model, according to the lesson in week 3 of the MLS Course 1, underfitting is identified when the model does not perform well on the training dataset nor the testing dataset. Overfitting on the other hand, performs well on the training dataset but poorly on the testing dataset.
How will I know it’s not performing well on the training dataset?
Hi there
you can see it by evaluating the performance metrics of your model after training. Let’s assume a regression model would solve your business problem in case a certain KPI is fulfilled (e.g. Temperatur error < 2 K for all predictions)
After the training you can evaluate your model performance, e.g.:
and analyse the residua. You can then compare the residua against your defined success criteria. (2K)
(note: while you can evaluate your training data multiple time, you are only allowed to use the test data for a final test. So we need to make sure no information from the test set makes it into our training or hyperparameter tuning process. This is why we often keep a third set, the validation data)
You might also want to take a look at performance metrics during training of course and compare the performance. Here you see an example for overfitting indicated that test loss is much higher than training loss, meaning the model does not generalise well:
This thread might be interesting for you:
Best
Christian
