Gradient Descent Graph

In the 3d model Andrew showed about gradient descent, he said that if we took the starting point a bit to the right we would end up in a different local minima. But what if started on the back of that hill, we end up in the plains, and not any minima. Does this ever happen?

Hello @anishgupta

I assume you are asking about this slide in the video Gradient Descent:

Yes - that situation can happen. In response to this case, we would need to train a few models starting from a few different positions with the hope that some of them will reach a better position.


1 Like

Thanks for the explanation @rmwkwok