I have listened and watched the video lesson entitled “Cost function for logistic regression” and at 9 mins 28 secs Andrew seems to have made a mistake here as he says “In fact, if f of x approaches 0, the loss here actually goes really large and in fact approaches infinity.”.
However, in this part of the lesson, Andrew is talking about the loss function:
-log(1 - f_{\vec w, b}(\vec x^{(i)}))
and what happens to its value when f(x) approaches 0. The loss function in fact approaches 0 too as f(x) approaches 0, not \infty.
The equations below also seem to contradict what he is saying as they say:
As -log(1 - f_{\vec w, b}(\vec x^{(i)})) \to 1 then loss \to \infty
Can anyone confirm that a mistake has been made here?
The are two scenarios when dealing with the loss function:
When the target label Y=1
when the target label Y=0
In the case when Y=0, and f(x) → 1, then the loss is → ∞. That is to say that when the model prediction is high, near 1, but the target label is actually 0, then the loss has to be very high, to reflect the inaccuracy.
Here is a screenshot of the lecture.
Yes, I withdrew my earlier reply after listening to his summary again. The content of the slide is correct, but he said it wrong by mistake, I guess it is a slip of the tongue. This matter is now lodged with a staff member who is taking a further look into it.
The MLS series of courses was published just over 2 years ago. You have a keen eye/ear for details and nothing escapes your attention. We appreciate very much your effort to help us in getting things right.
