Is there any decision boundary when there is one feature in case of logistic regression?
a) Is the line passing through the threshold point, or y = 0.5, a decision boundary?
When there are multiple features, then the decision boundary concept is clear to me, as Prof. Andrew taught in the lecture. For example, if x1+x2>=3 is obtained, we will take x1 as the x axis and x2 as the y axis, then just draw the equation line, which is the decision boundary.
But I am really confused: when there is a single feature, what is the decision boundary? for example:
g(z)>=0.5
z>=0
wx+b>=0
suppose I get by placing w and b values: x>=-2,
b) So will this equation be the decision boundary in the plot g(z) vs. x~single input?
Yes, though it would be more of a decision line, rather than a boundary. This is because with only one one feature, you can only make a 2D plot where ‘x’ is on the horizontal axis, and the y and prediction values are on the vertical axis.
With y and the predictions on the vertical axis, the boundary would be a horizontal line at 0.5.
When x = 0, g(z) = 0.5738, but when x = -0.3, g(z) should be 0.5, so it would be better if you redraw B such that the steepest slope is at x = -0.3.
You see why g(z) = 0.5 always has the steepest slope?
In your A, you have drawn the boundary horizontally, but drawing it vertically is also fine, and the same applies to B too.
If you discuss the boundary with respect to x, that will be a vertical line in your B; if it is with respect to z, that will be a vertical line in your A; if with respect to g(z), it will be a horizontal line in either A or B.
All boundaries above are lines because we are drawing them on a 2D plane. If we have only the feature axis (without g(z)), then it is going to be a boundary point. So, one feature → boundary point; two features → boundary line; three features → boundary plane.
The challenge with visualizing this boundary is that even though there is only one feature x, there are still two learned parameters: w and b.
So if you plot x on the horizontal axis, and y on the vertical axis, you have to assume some constant values for w and b. But the exact w and b value will shift the result of the sigmoid (vertically). And unless you have completed training, w and b are both variables.
So this really is a complex system to plot in general.
This is very difficult to visualize, because you either have a complicated boundary plane in 3D, or you have to use markers for the ‘y’ values, but you cannot use markers for the g(z) values, because they are real numbers (not just labels).
Tom’s comment reminded me of this, although it looks totally fine to me you labeled your y axis in B as g(z), another way is to label it as g(wx+b) so that we can see that it is a function of x.
However, as I said, you might want to shift your sigmoid curve on B a bit so that the steepest slope is at x = -0.3.
