I have visualized the sigmoid function in 1-d feature logistic function and it gives the S shaped curve which is then used to fit on the two classes. It made sense to me but when I had seen the example with more than 1 feature, it is getting difficult to understand, where is the shaped curve here
As a general decision, I struggle with visualizing things in higher dimension, please help me with this as well. How do you guys visualize and understand problem in higher dimensions like 3D.
Hello @tbhaxor,
When you have one feature and you represent it with the x-axis, you can draw the sigmoid curve by putting the output of the sigmoid function on a second axis which is the y-axis.
When you have two features and you represent them with the x- and the y- axes, you can draw the sigmoid curve by putting the output of the sigmoid function on a third axis which is the z axis. (I found this by googling).
When you have three features and you represent them with the x-, y- and the z- axes, since you can’t draw the 4-th axis, you can’t put the output of the sigmoid function on it. This is a limitation.
The plot that you shared adopted a different strategy. Instead of showing the whole sigmoid curve, it shows only where the sigmoid function outputs 0.5 (boundary line). Therefore we don’t need the 3rd axis in the case of two features.
If you adopt the same strategy on the case of three features, you can also only show where the sigmoid function outputs 0.5 (boundary plane) to avoid adding the 4th axis.
However, when you have more than 3 features, we can no longer make a plot that even just shows the boundary line.
You will need to figure out some other ways to explain your model and there is no general method for that.
Cheers,
Raymond
So here sigmoid function i used to find “that threshold” of the “decision boundary”?
The idea is to sample your x-y space and check out at which points do your model return 0.5, then connect those points up to form the boundary line.
Yes it makes sense, mathematically that would be setting \frac{1}{1+e^{-z}} = 0.5 \implies z = 0 and we know that z = \vec W \cdot \vec X + b
Please correct me if I am wrong.
You are right!
Raymond
It is like taking a metallic sheet and beating it enough to have a S shaped figure 
Well, you know the sigmoid function is to “bend” values that are too large to 1, and “bend” values that are too negatively large to 0. Sometimes I think it’s like “hammering” those data points to where they should stay quietly. Haha 
Raymond
