Cost function. linear regression

Prof. Andrew said in the explanation of the contour plot that for different values of w and b, there can be the same cost function.Can you please explain this a little bit?
I suppose that the hammock shape of the cost function plot is symmetric, and so it has 4 points that have the same cost function. Among those points, every two points have either the same w or the same b. lol. I don’t know if I said something wrong or completely weird.

Hi @farhana_hossain,

I believe by “same cost function”, you meant “same cost value”.

Let’s discuss a linear problem y = f(x) optimizing for the squared error.

If f(x) = wx, then the cost function is a function of only 1 weight, and it is a parabola, and for each valid cost value, it can be achieved by two possible weight values.

If f(x) = wx + b, then the cost function is a function of two weights (w and b), and it is a paraboloid, and for each valid cost value, it can be achieved by an infinite number of possible weight pairs. Those pairs, when connected, form a contour line. If you find it hard to imagine, ask yourself this: on a mountain of 100m tall, how many possible locations are at the height of 25m?

Note the difference between a parabola and a paraboloid. You should see paraboloids in some lecture videos and therefore explain the origin of contour line. Please watch those lectures again.

Raymond

1 Like

Got it. I imagined a soup bowl now, and yes, there are many points, not just 4.