How would it not matter when the train, dev, and test sets aren’t having the same distribution?
Isn’t in this example, we are training the model on wrong-labeled data?
- coursera-platform
- dl-ai-learning-platform
1 Like
The test set, your model never sees, it only seen in inference or after training has ended so you don’t really know what’s in there.
You do hope and you monitor data of test set that they have the same distribution with the validation and dev sets but it won’t be as perfect as those otherwise what’s the predictive capability of your model.
Ideally you would want to have validation and dev sets with similar distribution although they should also contain small differences to prepare the model for further deviations in real life test set inferences.
1 Like