Hyperparameters Tuning - Panda

Stupid question here !!! in the course “panda vs caviar”, it is mentioned that, for the babysit approach, you can start on day 0 with parameters initialized at random then on day 1 you may change the learning rate if everything goes well.

OK but what does it mean practically in the Jupyter Notebook ?
Do you use any ‘stop’ button then replace your learning rate and restart the cell (don’t we loose the training results obtained so far ) ? Or does it mean that you completely stop your current training and restart from scratch ?



1 Like

Hey @Michael_Brion , firstly, this is not a stupid question.

Secondly, you can change the learning rate programatically. Like, for example, after 100 epochs you can do something like learingRate = learningRate/10

Another way would be as you mentioned, you test out using different learning rates. You can have different model trained based on different learning rates and then compare them for results. You’ll learn (theoretically) more about it it Course 3.

1 Like

Hello Mubsi, hope you are doing well.

You know, I also felt a bit silly when I stumbled upon this “Pandas VS Caviar” analogy… "Where is my college-level math??? :thinking: I don’t understand this extremely-simple analogy!!! :grin: "

Now seriously: In the example provided by Andrew, when he said “change one hyperparameter a day”, was he (probably) referring to those very-deep models that literally take days to train? (And so what he said would make sense)… Is my assertion correct?

Plus, if we think about the “Caviar” training approach, that one is computationally more expensive than the “Pandas” approach (provided the same “depth” model is being analyzed in both cases), isn’t it?

Thank you very much!