Is Logistic Loss Function based in probability and inverse exponent?

Course Q&A Machine Learning Specialization Supervised ML: Regression and Classification
1

In one of the lectures, Dr. Ng mentioned f_w,b(x) = P(y=1|x;w,b), which is a conditional probability. I’m going to simplify and call it f from now. If the output is a probability, it implies 0<=f<=1. We take it to the Logistic Lost Function where the Loss(f) has two cases:
if y=1, Loss(f) = -log(f)
if y=0, Loss(f) = -log(1-f)

  • if f is a probability, then by complement’s rule, P(A’) = 1-P(A). and P(A’)+P(A)=1.
  • we’re inversing the g(z) sigmoid function using logarithm so we don’t have to deal with euler’s number but the exponent z.

I’d like to know if my understanding is correct.

2

I’ll say your understanding of loss function in the context of logistic regression is super on point. But let’s wait for actual mentors to comment.

1 Like
3

The cross entropy loss function -log(\hat{y}) is not the inverse function of sigmoid. I also don’t know what you mean by avoiding Euler’s constant here: note that log means natural logarithm in Machine Learning, so e is involved.

Here is a thread from mentor Raymond that explains how the loss function works and why it is defined the way it is.

Here’s another thread that includes a graph of log(z) which will help with the intuition.

1 Like
4

Thanks, I misunderstood the meaning. The threads you mentioned are also very helpful. I’ll have to learn about maximum likelihood as well.