Hello everyone!
I am writing a short discussion topic as I was wondering why the methods of cross-validation (both nested and non-nested) are not included in the course section about hyperparameter tuning. I had heard of it in a previous course on Machine Learning during my studies, and was curious to understand why it is not mentioned in this course.
My thought would be that it is less relevant for Deep Learning because it is computationally too expensive, but perhaps there are other reasons?
If anyone knows more about the topic I’d be curious to hear about it!
Cheers,
Cross validation is used exactly in the case of hyperparameter tuning. Prof Ng does cover that in Week 1 of DLS C2 (see the very first lecture in W1). He also covers it in even more depth in DLS C3. In the section where he discusses how to search a large space of hyperparameters in C2 W3, he may not specifically mention cross validation, because he’s assuming you already know about that from what he said in W1.
2 Likes
We have already discussed this topic a few times here as I remember. The main reason I think he doesn’t focus on CV is because its inefficient for large amounts of data especially when training neural networks which need comparatively speaking a lot of data, to be properly trained…
1 Like