My doubt is in Linear regression we have equation like y=wx+b and it’s graph is a simple line on a 2-d plane. But if we try to visualize it with multiple parameters (weights) let’s say just 2. It’ll be like y = w_1.x_1 + w_2.x_2 + b. But it’s graph is something like a plane in 3-d. As below and it is not linear anymore.
At this point my question is How is it linear ? As with just two parameters it’s graph changes from just simple line in 2-d to a plane in 3-d.
Now, on the same note if we talk about the cost function. In the lecture it is said that It always going to have a global minima. But there also with single parameter, graph is like a parabola and I am sure there is a single minima as I can see it and with two parameters it is like a convex function with a single global minima also.
But what is the surety of it having a global minima with more than two parameters ?
The cost function for linear regression turns out to be convex. You can have many more than 2 parameters for a linear regression problem and the cost function will still be convex with a single global minimum. Although as Tom points out, it may not be very easy to visualize the cost function surface in the case of more than two parameters.
@paulinpaloalto
Okay, Sir but is there any mathematical theorem that proves it ? What is it based on ? How are you sure that there will always be a single global minima ?
