Gradient Descent starting at the "top of the hill"

Course Q&A Machine Learning Specialization Supervised ML: Regression and Classification
1

Always exciting to revisit these fundamental concepts as we look at things differently.

I was curious about the starting point for Gradient Descent when you are on top of the hill as explained in the video.
How do you begin taking that “baby step” with the fastest drop when you are starting on top of the hill and all immediate directions are equal?

1 Like
2

I think that if all the immediate directions are equal then we are at a local maxima, where the derivative is 0 and we are stuck at that point, in reality when we initialise weights randomly there is a very less chance we encounter a local maxima, most of the times we are not actually on the top of the hill but somewhere in between.

Even if we get stuck at local minima, we may run the model many times hoping it would reach global optima in any one of those.

correct me if anything’s wrong :smile:

2 Likes
3

I see. It’s the random initialization of weights that I missed if I’m not mistaken. If that’s the case then it makes sense.
Thanks.