Is Logistic Loss Function based in probability and inverse exponent?

In one of the lectures, Dr. Ng mentioned f_w,b(x) = P(y=1|x;w,b), which is a conditional probability. I’m going to simplify and call it f from now. If the output is a probability, it implies 0<=f<=1. We take it to the Logistic Lost Function where the Loss(f) has two cases:
if y=1, Loss(f) = -log(f)
if y=0, Loss(f) = -log(1-f)

  • if f is a probability, then by complement’s rule, P(A’) = 1-P(A). and P(A’)+P(A)=1.
  • we’re inversing the g(z) sigmoid function using logarithm so we don’t have to deal with euler’s number but the exponent z.

I’d like to know if my understanding is correct.

I’ll say your understanding of loss function in the context of logistic regression is super on point. But let’s wait for actual mentors to comment.

1 Like

The cross entropy loss function -log(\hat{y}) is not the inverse function of sigmoid. I also don’t know what you mean by avoiding Euler’s constant here: note that log means natural logarithm in Machine Learning, so e is involved.

Here is a thread from mentor Raymond that explains how the loss function works and why it is defined the way it is.

Here’s another thread that includes a graph of log(z) which will help with the intuition.

1 Like

Thanks, I misunderstood the meaning. The threads you mentioned are also very helpful. I’ll have to learn about maximum likelihood as well.