Optional Logistic Regression: Cost Function

In optional logistic regression cost function at the start of the Math Derivation paragraph we read:

“From the above, you can see that when y=1, you get h(x(i),θ), and when y=0, you get (1−h(x(i),θ)), which makes sense, since the two probabilities equal to 1. When y=0, you want (1−h(x(i),θ)) to be close to 1, which means that h(x(i)) needs to be close to 0, and therefore h(x(i),θ) close to 1. When y=1, you want h(x(i),θ)=1.”

Shouldn’t it be when y =0 you want (1-h(x(i),θ)) to be close to 0 so the final predicted probability is 0 when the actual class is 0?

Hi @Iraklis99

No, the way it is formulated is correct:
When y=0 (the actual label is 0), you want (1−h(x(i),θ)) to be close to 1, because if you predicted correctly (that is h(x(i),θ) is close to zero), then log of (1-0) is zero which means your loss is minimum.

" When y=0, you want (1−h(x(i),θ)) to be close to 1 , which means that h(x(i)) needs to be close to 0, and therefore h(x(i),θ) close to 1 "

Here it says h(x(i),θ) needs to be close to 1 when y=0, is this correct, shouldn’t it be 0?

1 Like

Yes, actually you are correct, I missed the second part - h(x(i),θ) needs to be close to 0. I will submit it for fixing. Nice catch :+1:

1 Like