In Deep learning course 2 Hyperparameter Tuning Why we taken -4 in that course video like
r= -4*np.random.randn()
alpha=10**r’
why we taken -4???
We want to sample learning rates in range [10^{-4}, 10^0) in the scenario presented in the lecture.
log_{10}(10^{-4}) = -4
Similarly, log_{10}(10^0) = 1
Learning rate is calculated using the formula 10^r.
np.random.rand returns values in range [0, 1). r falls in range [-4, 0).
Based on the above value of r, the learning rate is now in range [10^{-4}, 10^0).
1 Like