Interpreting logistic regression - prediction and error on sigmoid plot

[neural_ghost]

Hi, I would like to ask

1. why the w0 and b values for the second dataset in Gradient Descent For Logistic Regression lab are such that the sigmoid function isn’t steep enough to pass through all data points:
   

What I mean is that I would have supposed if the blue line sigmoid(z) passed through 3rd and 4th point here (and was therefore much more steep), the probability would’ve been more precise.
Something like this:

(sorry for a hand drawing in purple)
The contour plot doesn’t let me click in the center of the smallest circle, but is my guess correct that here the cost would be nearly 0 and the sigmoid would be as the purple line on the graph above?

2. I am trying to understand what is the meaning of the cost for point that the sigmoid doesn’t pass through, for instance if the data points were as such:
   
   
   
   (I managed to “reverse” the prediction!)

Do I understand it correctly that

• the cost for a point (error) on below graph would be as I marked with purple, for the reason that for such Sigmoid blue line, the total loss is minimal, even tho still big?
• and that for below data, it is impossible for the model to have a zero loss?

Full plot for reference:

3. And lastly, do I understand it correctly that the orange line z is the decision boundary but plotted on sigmoid graph?

I think understanding it well would help me to understand the interpretation of the math in the area where it is unclear whether a tumor is malignant or bening, even for a professional doctor. I’d greatly appreciate help here.

[rmwkwok]

1 ) Because the model isn’t trained long enough.

2a) the length of those purple vertical lines are the sizes of errors
2b) impossible - you can’t draw a sigmoid that passes through all red and blue data points, can you?

3 ) the orange line is y = 4.79x - 11.77.

[neural_ghost]

Ok, thank you, that clears it up mostly.

I still have some doubts though:

1. by not trained long enough, do you mean the values of \vec{w} and b aren’t precise enough?
2. But what is the orange line for, then? I’m afraid I didn’t catch it. I mean I can see the values of w and b for this f(x), and it’s the “z” of g(z). So the sigmoid function transforms the orange line into the blue S-shaped line. But what is the interpretation of z, the orange line? As in, blue sigmoid is the probability a tumor of certain size is benign or malignant, after having decided on a threshold. And analogically, what would be the interpretation of z on this graph?

[rmwkwok]

1. if you train it for more iterations (rounds), you will get w and b that’s at smaller cost
2. it’s a x-y plot (watch the axis name), so I said the orange line is y = 4.79x - 11.77. I would say the blue line is y=sigmoid(4.79x-11.77).