In a gradient descent algorithm, do we consider global minimum or local minimum?
How do we know whether our model is at local minimum or local minimum?
Thanks
Gradient descent can’t tell the difference.
That’s why most cost functions are convex - they only have one minimum.
1 Like
I don’t think any model could ever know where the global minimum is located, even if its sitting on it, otherwise it will be able to perform perfectly.
The whole point of training (a trial and error process) is to establish a minimum, by chance could be the global also!
1 Like
It might help us to know what courses you are taking here or have taken, so that we understand the context of your question. Here’s a thread from a couple of years ago that discusses this point in some detail based on a question that was asked in the context of the Deep Learning Specialization (DLS) Course 1. It also links to a couple of other interesting topics like Weight Space Symmetry, which shows that the number of local optima is extremely large when you’re dealing with Neural Networks.