In the 3d model Andrew showed about gradient descent, he said that if we took the starting point a bit to the right we would end up in a different local minima. But what if started on the back of that hill, we end up in the plains, and not any minima. Does this ever happen?
Hello @anishgupta
I assume you are asking about this slide in the video Gradient Descent:
Yes - that situation can happen. In response to this case, we would need to train a few models starting from a few different positions with the hope that some of them will reach a better position.
Raymond
1 Like
Thanks for the explanation @rmwkwok