Saw this concept in the Week 2 Assignment, but could not fully understand it.
What that means is that there is a “geometric” interpretion of what Logistic Regression is doing: it is finding a hyperplane in the input space that divides the “yes” answers from the “no” answers. We are solving for the coefficients and bias of a linear transformation that gives the minimal cost. Once we have w and b, the hyperplane is defined by
w^T \cdot x + b = 0
Of course there is no guarantee that there is such a hyperplane that perfectly divides the samples in that way: it all depends on what the data actually is. That’s why LR gives us only 70% accuracy on the image recognition task in Week 2. Neural Networks will allow us to represent much more complicated decision boundaries and will thus give better results on this type of task in general.
6 Likes
Wow, great to know. Thanks Paul. 
Sometimes I just could not find definitions for these kinds of concepts and thus felt they were hard to understand.
Adding a little to what Paul said: even if there is no hyperplane that perfectly splits the samples, there may be one (or several) that helps: maybe one side is all one way, or there is a split where each side will be “close” to purely one way.
When we get to multi-layer networks it can help to think of the first layer as being a variety of such helpful splits that are giving information, and the next layer is combing the first level distinctions.
3 Likes
Adding to @paulinpaloalto excellent answer, you can also obtain a graphical representation of how a Linear Regression splits the space between “yes” and “no” answers using the Tensorflow Playground, setting up a NN with just 1 Neuron and letting it run for some epochs.
In the big square at the right, the Blue and Orange Backgrounds represent the space split generated by the linear regression
You will notice that the split is far from perfect, i.e. there are a lot of Orange points in the Blue space, and some Blue points in the Orange space. This is a limitation of the linear regression.
Tensorflow Playground
6 Likes
Thanks for the remarks 
it is nice to visualise some concepts and remarks here. Thanks for the info.
1 Like