I know the purpose of training and testing data
I did not understand why we separate 20% or whatever of data for validation
Is it very important?
what does that mean?
Hi, @IZZETTIN_ALHALIL. To my knowledge, it is a fairly arbitrary and based on the accepted wisdom of practitioners–kind of a “folk theorem.” There is also a 60-20-20 practice for training/validation/and test. The validation set is used for hyperparameter tuning to help avoid “fishing expeditions.”
You have posted this in the Course 1 forum. Be aware that there is more discussion of these issues in Course 2 and 3.
By a fishing expedition, I mean the an exercise in which one is trying to make the test results look good, probably to satisfy a critic, or worse, to fool one’s self. Best to keep those two exercises separate.
sorry for posting here, I was confused
I appreciate your comments
thanks