Hello everyone,
In the slide, when the threshold rise, precision will get higher. But I don’t know where my “count-example” is wrong, can anyone help me? PS:“X” denotes actual negative, "O"denotes actual positive
@13695789309 Looking at this, and someone else can correct me if I am wrong, but recall the threshold is used to determine the outcome of each prediction, either zero or 1.
It is not however serving as a sort of ‘bounding box’ over your total set of predictions to be used when calculating precision. Or, in both cases your true positives / total predicted positive are the same, 4/5-- But, when you ran the single prediction on X, the threshold was such that it mispredicted a positive for an actual negative.
The threshold applies at the level of individual predictions, not across the set of all predictions.
thanks for your reply,
but sorry, I can’t understand What the difference between
using threshold to predict y hat of a series of input one by one
and
using it predict one input’s y hat.
I think your drawing here is confusing you.
You’ve given 5 predictions. At the individual level of each prediction, your threshold level determines whether each of the 5 predictions is either an X or an O– However the total number of predictions never changes.
The way you’ve drawn the threshold is as if it were being applied at the set level of all predictions (i.e. as if you were tossing certain predictions out, whether or not they were correct). This is not how it works.
Hello, @13695789309, and @Nevermnd,
I think we are focusing on just the first 4 predictions because precision requires us -
- true positives ( True positives above the threshold)
- total predicted positive (Samples above the threshold)
Even though we make predictions for all samples, but from the above two points, we see that, to calculate precision, we concern only those above the threshold.
As for why threshold up and precision down, I would only say that the slide did not mean to establish any monotonic relation between them.
The slide only means to talk about precision-recall tradeoff. The curve in the lower right by no means tell us strictly that the threshold value must be monotonically changing along the curve.
With your example, @13695789309, we can see the precision-recall tradeoff.
Cheers,
Raymond
the denominator of precision is the number of actual and predicted positive example, so the precision will change after rising threshold. Is it right?
Hello,thanks for your reply.
I still have a question.
In my example, the recall also get lower as threshold rising(from 1 to 0.75).It seemingly not exist trade off between precision and recall
I think there are some problems in my example 
No, there is no problem with your example. We just need to try to understand it 
The green shift (indicated by the green arrow) sees higher precision and lower recall. Right? That’s the trade-off.
The green shift rules out at least one wrong prediction, but the blue shift rules out no wrong prediction. Was it wise to do the blue shift at all? Doing it does not give us anything good, only giving up a good prediction. That is why we should not expect any improvement from the blue shift in any metric, should we? That is not something we will expect to do, is that?
To really draw a smooth tradeoff curve like the one in the lower right of the slide, we are dealing with many samples (in contrast to just five in your example), and we are changing the threshold in larger steps so that each step will cover quite some samples (in contrast to covering just one as in your example). With larger steps, it is quite likely that, as we move the threshold up, we do not just rule out good predictions.
Cheers,
Raymond
1 Like
Thanks a lot 
So we should recognize trade off as a whole picture rather than a regional step, is it right?
1 Like
The whole picture is a good way of considering it, the key is not to consider it in sample-by-sample steps.
As long as we can rule out one wrong prediction (false positive), we have precision increased, and if that also rules out one correct prediction (true positive), we have recall decreased. That takes at least two samples.
Cheers,
Raymond
1 Like
Your explanation is great! 
I completely understand. Thanks again!
1 Like
You are welcome!
Note,I deleted my previous reply, because it did not apply to this topic.
Sorry, I hadn’t gone through this exact lecture, so I myself was misunderstanding here.
My thought was, originally, obviously we cannot ‘juice’ the model, which seemed my concern at first when you were seemingly tossing out prediction points based on the threshold.
But, we can alter the way we are interpreting the results. In just an ‘intuitive’ way in my mind, this is similar to applying a certain sort of ‘confidence level’, depending on whether we are focusing on precision or recall.
But I see now my ‘per-prediction’ assumption was incorrect, sorry about that.
At that level, I mean in truth I think you would actually have some affect-- I mean if the probability of prediction is .5, you are kind of 50/50 either way. But if you increased your prediction threshold to say .9, so a result has to have a 90% or above probability to be a positive, you are very going to very strongly bias your model to only classify O's when you are ‘really, really sure’, at the risk of maybe missing some cases that have valid ground truths, but your model finds less probable.
But I guess that was not the question you were asking.
2 Likes