Gradient Descent Algorithm Doubt

[paulinpaloalto]

It might help us to know what courses you are taking here or have taken, so that we understand the context of your question. Here’s a thread from a couple of years ago that discusses this point in some detail based on a question that was asked in the context of the Deep Learning Specialization (DLS) Course 1. It also links to a couple of other interesting topics like Weight Space Symmetry, which shows that the number of local optima is extremely large when you’re dealing with Neural Networks.