C2 Theoretical Questions

Course Q&A Deep Learning Specialization Improving Deep Neural Networks: Hyperparameter tun
1

I have some questions on theory, I’d like to raise them in one topic, if you agree.

  1. Hybrid Minibatches. Is it reasonable to make a mixed minibatch-iteration approach? In example I could split 10k samples data into 10 batches of 1k, and iterate training with these batches 100 times. Or even more complex - iterate 10-100 times with one minibatch, and than switch to another. Is it a reasonable approach?
2

No. You will overfit to the mini batches you use the most. It is likely your model will not generalize well.