How we select learning rate

In Deep learning course 2 Hyperparameter Tuning Why we taken -4 in that course video like

r= -4*np.random.randn()

why we taken -4???

We want to sample learning rates in range [10^{-4}, 10^0) in the scenario presented in the lecture.

log_{10}(10^{-4}) = -4
Similarly, log_{10}(10^0) = 1

Learning rate is calculated using the formula 10^r.
np.random.rand returns values in range [0, 1). r falls in range [-4, 0).
Based on the above value of r, the learning rate is now in range [10^{-4}, 10^0).

1 Like