Thank you for the lecture. I would like to ask about the possibility of having a dev error higher than the training, is it possible ? if it is possible what does it mean? and how to solve such a thing ?
Hi Mohammed,
It’s very common to have the dev error higher than the training error. Training error is likely to be the least of the 3 errors: training, dev, test, since model has already seen the training samples and tried to learn from it.
Model never trains on dev/test data so error will likely to be higher there.
Normal training process should take of the difference in error between training and dev set. As the training error falls, usually dev error will also fall to a certain extent, till it starts to overfit on the training data. You can try out different network architectures, regularization techniques or callbacks like early-stopping to prevent overfitting.
Hope this helps.
Please note that the cases that Somesh describes are also covered by Prof Ng in some detail in the first series of lectures in Week 1 of DLS Course 2. So please “stay tuned” and listen to all that he has to say about what it means when the training error is less than the dev and/or test errors and how to approach solving those issues.