Hi,
In lab04, gradient descent, we saw how the gradient descent algorithm works with a sample dataset. In this example, we used an initial learning rate = 1.0e-2 and iterations = 10000.
I’m aware of the implications of selecting a value for alpha that is too large or too small. However, is there a rule of thumb that could be applied when choosing that initial value? Similarly, what about the number of iterations, 100, 1000, 100000? What would you advise?
Regards
No, there is not.
Pick an initial number, and plot the cost history. If the cost hasn’t settled by the end of the iterations, increase the learning rate,or increase the number of iterations.
If the cost hasn’t settled by the end of the iterations
Follow-up question. How do I know when the cost settles?
I’m starting week 2 today, so I’m unsure if this will be covered in the upcoming videos or if I completely missed it from the previous ones.
It’s a cost history plot.
It is a plot with current cost value (on the vertical axis) vs the iteration number (on the horizontal axis). As the model converges, the curve goes flat.
Or you can look at the changes in the weight values that the model is using. When the values are no longer changing significantly, the model has converged.