Gradient Descent local max

[A_R_1]

Hi,

I had a question about a local max on a graph. Imagine you choose a large learning rate such that it keeps overshooting and moving away from the local minimum. Assume you somehow reach a local maximum such that the slope of the graph is 0. Would your gradient descent equation not give w = w and tell you that you found your local minimum?

[TMosh]

There are no local maxima, because the cost function is known to be convex.

[Darvin_Cotrina]

Hi @A_R_1

Remember when you’re using a square error cost function with linear regression, the cost function doesn’t and will never have multiple local mínimum (it’s a convex function)

[TMosh]

The same can be said of the logistic cost function - it’s also convex.

[Darvin_Cotrina]

Ok, I try to explicate
in square error cost function, is a convex

J(w) = \dfrac{1}{2m} \sum_{i=1}^{m} (h_w(x^{(i)}) - y^{(i)})^2

in logistic cost function, is not lineal and is not convex
.

J(w) = -\dfrac{1}{m} \sum_{i=1}^{m} [y^{(i)}log(h_w(x^{(i)})) + (1 - y^{(i)})log(1 - h_w(x^{(i)}))]

I hope I have helped

[TMosh]

The logistic cost function is convex. You have not draw the correct curve.

It doesn’t have to be linear or use squares to be convex.