Stupid question here !!! in the course “panda vs caviar”, it is mentioned that, for the babysit approach, you can start on day 0 with parameters initialized at random then on day 1 you may change the learning rate if everything goes well.
OK but what does it mean practically in the Jupyter Notebook ?
Do you use any ‘stop’ button then replace your learning rate and restart the cell (don’t we loose the training results obtained so far ) ? Or does it mean that you completely stop your current training and restart from scratch ?
Hey @Michael_Brion , firstly, this is not a stupid question.
Secondly, you can change the learning rate programatically. Like, for example, after 100 epochs you can do something like learingRate = learningRate/10
Another way would be as you mentioned, you test out using different learning rates. You can have different model trained based on different learning rates and then compare them for results. You’ll learn (theoretically) more about it it Course 3.
You know, I also felt a bit silly when I stumbled upon this “Pandas VS Caviar” analogy… "Where is my college-level math??? I don’t understand this extremely-simple analogy!!! "
Now seriously: In the example provided by Andrew, when he said “change one hyperparameter a day”, was he (probably) referring to those very-deep models that literally take days to train? (And so what he said would make sense)… Is my assertion correct?
Plus, if we think about the “Caviar” training approach, that one is computationally more expensive than the “Pandas” approach (provided the same “depth” model is being analyzed in both cases), isn’t it?
I don’t understand this extremely-simple analogy!!! 
"