# Cross validation? **URL:** \ **Category:** Structuring Machine Learning Projects\ **Tags:** coursera-platform\ **Created:** [July 5, 2021, 12:48am UTC](https://community.deeplearning.ai/t/cross-validation/19154 "2021-07-05T00:48:39Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![michael.brent](https://avatars.discourse-cdn.com/v4/letter/m/b77776/32.png) [@michael.brent](https://community.deeplearning.ai/u/michael.brent)\ **Post date:** [July 5, 2021, 12:48am UTC](https://community.deeplearning.ai/t/cross-validation/19154/1 "2021-07-05T00:48:39Z")
I have always used cross-validation for training and testing, which seems much more efficient than using a true dev set. Do you think cross-validation is just as good, or do you recommend against it? If you recommend against it, why? ---
**Author:** ![carloshvp](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/carloshvp/32/68_2.png) [@carloshvp](https://community.deeplearning.ai/u/carloshvp)\ **Post date:** [July 5, 2021, 6:36am UTC](https://community.deeplearning.ai/t/cross-validation/19154/2 "2021-07-05T06:36:04Z")
Hi @michael.brent , this is a very interesting question. In the end it depends on what is your objective. With cross-validation you are training several times, leaving out each time a different part of the dataset (the test subset). At the end you probably make an average and std deviation of the metrics to conclude how good the model generalizes. Using a single train-dev-test iteration will be faster in training (only once!) but you have the risk of being misled by the selected dev and test sets. Here a nice picture showing cross-validation with all their iterations > **[Cross-validation (statistics)](https://en.m.wikipedia.org/wiki/Cross-validation_(statistics)#/media/File:K-fold_cross_validation_EN.svg)** > > Cross-validation, sometimes called rotation estimation or out-of-sample testing, is any of various similar model validation techniques for assessing how the results of a statistical analysis will generalize to an independent data set. It is mainly used in settings where the goal is prediction, and one wants to estimate how accurately a predictive model will perform in practice. In a prediction problem, a model is usually given a dataset of known data on which training is run (t One round of cros... ---
**Author:** ![carloshvp](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/carloshvp/32/68_2.png) [@carloshvp](https://community.deeplearning.ai/u/carloshvp)\ **Post date:** [July 10, 2021, 12:38pm UTC](https://community.deeplearning.ai/t/cross-validation/19154/3 "2021-07-10T12:38:50Z")
Hi @michael.brent , did my answer help in any way or did I misunderstand you? ---
**Author:** ![michael.brent](https://avatars.discourse-cdn.com/v4/letter/m/b77776/32.png) [@michael.brent](https://community.deeplearning.ai/u/michael.brent)\ **Post date:** [July 10, 2021, 2:05pm UTC](https://community.deeplearning.ai/t/cross-validation/19154/4 "2021-07-10T14:05:52Z")
You provided some information on cross-validation but I already knew that. I use cross-validation all the time. What I was able to take away from your response is that there is no advantage to having a single dev set except for speed (fewer training runs). Is that correct? The thrust of my question had to do with overfitting. If you iterate on an algorithm for months, even if the algorithm doesn’t train on the dev set (or if you use cross-validation), you are manually making choices that cause the algorithm to work well on the dev set and hence you have the potential to overfit to it (by human decision-making or parameter tuning). My question was whether a single dev set has any advantages with respect to CV in this regard. In particular, if I am doing CV, do I still have to have a separate dev set that is not part of the CV to avoid the overfitting of human choices to the the hold-out sets in the CV. I understood you to be saying “no”, that CV fulfills the role of the dev set and the test set will reveal whether overfitting to the dev set by human decisions occurred. ---
**Author:** ![carloshvp](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/carloshvp/32/68_2.png) [@carloshvp](https://community.deeplearning.ai/u/carloshvp)\ **Post date:** [July 11, 2021, 7:36am UTC](https://community.deeplearning.ai/t/cross-validation/19154/5 "2021-07-11T07:36:38Z")
hi @michael.brent , cross-validation is a more robust validation than single dev-test set. This is the advantage. But as you mention that you have algorithm iterating for months, you might want to consider not using cross-validation during the exploratory phase to speed your training, possibly 4-5 times or more depending on your k in k-fold cross-validation. For example, if you are currently running a training algorithm which takes 1 month with 5-fold CV, you can iterate 4-5x faster with single dev-test validation and try other algorithms with other hyperparameters in the same amount of time. When you find the right algorithm + hyperparemeters, maybe it is time to do cross-validation to re-confirm the metrics achieved. But this might not be needed at all, depending on your use-case ---
**Author:** ![carloshvp](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/carloshvp/32/68_2.png) [@carloshvp](https://community.deeplearning.ai/u/carloshvp)\ **Post date:** [July 18, 2021, 2:50pm UTC](https://community.deeplearning.ai/t/cross-validation/19154/6 "2021-07-18T14:50:04Z")
Hello @michael.brent , I hope we got to a common understanding or you didn’t see my previous message? Thanks a lot and enjoy the rest of Sunday!