This is in reference to logistic regression loss function video
Why does log y(hat) has to be a large number? And why should -log y(hat) should be a large negative number in the case of y=1? In the case of y=1, -log y(hat) should be close to 0 which means y(hat) should be close to 1. I really really don’t understand the suggestion of -log y(hat) being a large negative number. that is not the right scale for lowering error (or loss). can someone explain this to me?
You’ve thrown a lot of ideas at us there. For starters where does it say log(\hat{y}) “has to be a large number”? If we are using something as a “cost” or “loss” then we want it to a positive number and our goal is to make it small, rather than large. The first step in all this is to be clear that all the y and \hat{y} values are between 0 and 1 (inclusive). So what is the graph of the log function between 0 and 1? Here’s a thread which shows that.
Actually here’s another pre-existing thread that does a more complete explanation of loss functions and “log loss” in particular.
Please have a look at those two threads and then feel free to ask more followup questions based on what you learn there.
Hi Paul,
What you explained in the first link (Logistic Regression Cost Function - #2 by paulinpaloalto) was the exact answer to my question. That is what I thought should be the case but the wording threw me off thinking I am understanding it incorrectly. I thought the video is saying that -log y(hat) should be a large negative number (going to -inf).
Thanks for clearing that so quickly. Appreciate your time
1 Like