As discussed in C1W1, precision and recall are used for evaluating ML model performance on skewed datasets.
I have been using these two metrics as well, but had a discussion recently about recall being a prevalence-independent statistic. In my understanding, recall is more consistent than precision. When we evaluate the ML model using several test sets, each of which with a different distribution of class label y, the recalls on these test sets are rather similar, but the precisions aren’t.
Is my understanding correct?
Thanks.
1 Like
Dear Giovanni,
Thank you very much for your question. As you already know, precision is calculated as:
True positive / (True positive + False positive)
We can use this measurement when the cost of false positives is high(er). As an example, in a spam filtering program, if our classifier classifies important emails of a user as spam, the user will lose important emails and thus false positives have a much higher cost for us.
On the other hand, recall is calculated as:
True positive / (True positive + False negative)
thus by employing recall we are trying to penalize false negatives. Just imagine that we are developing a model which tries to detect patients who suffer from cancer. In this case, classifying someone who is suffering from cancer as a healthy person has an extremely high cost, thus here we use recall instead of precision.
In both cases the classes are unbalanced and the correct use of a metric is highly dependent on our use-case.
If you are merely try to have a better understanding of your classifier trained with an unbalanced dataset you could also use True positive or negative rate. Using this link you can read more about it 
Have a lovely day!
Kiavash
1 Like