Is error in option?

A bit confused, does second part of equastion is not 0?
Or is this error because we can’t fetch log 0?

The term you point to will be 0 in the case that y^{(i)} = 1, right? So that’s not the question here. It is the first term in the loss that matters and note that \displaystyle \lim_{z \rightarrow 0}log(z) = -\infty.

so, is the problem, that we can’t fetch log 0, right?

What do you mean by “fetch log(0)”? It is either undefined or -\infty depending on your nomenclature. But the point is that the h value is the output of sigmoid, which can never be exactly 0. But if it gets close to zero the log(h) value will be very large negative number, so the loss will be a very large positive number.

This is exactly the same loss function that you saw in DLS Course 1 Week 2, right? It works the same way here.

Your upper-right arrow is pointing to a term that is going to be zero anyway, regardless of the h(…) value, because y(i) is given as 1, and (1-1) = 0.

The significant term is the left one, which uses y(i) * log(h(…)).

can’t h(…) be 0?

Not technically, because it is the output of sigmoid, right? The output of sigmoid can never be exactly 0 from a mathematical perspective. Or exactly 1. But it can “round” to 0 or 1 in floating point.

And that was mentioned in the discussion earlier on this thread, right? Also note that your graph of the log function is way way too low resolution to be useful for our current purposes. We are only concerned about the domain of the function between 0 and 1, because the h() values are sigmoid outputs. Also note that you are graphing log_{10}, but we are using natural logs here. The fundamental shape of the curve between 0 and 1 is the same though.

Here’s a better version of the graph of the natural log function showing only the domain we care about:

That graph is from this other thread which discusses basically the same point.

1 Like