Vanishing gradient problem

No, as soon as you graduate from Logistic Regression to real Neural Networks, the cost surfaces are no longer convex and there are lots of local optima. Prof Ng will discuss this just briefly here in Course 1, but it turns out that this does not really cause that much of a problem. Here’s a thread which discusses this in a bit more detail and gives some links to relevant papers.

1 Like