Module2 Lab: Retrieval metrics: Recall: false negatives calculation unclear

Course Q&A Retrieval Augmented Generation
1

Hi.

Module2 Lab: Retrieval metrics:
The false negatives calculation in the given code works out to
true_positives - true_positives, which will always be 0. Therefore, the Recall will always be 1.0

What am I missing?

I am referring to the following code snippet in the jupyter notebook:

false_negatives = sum(
             newsgroups_train.target_names[df.iloc[idx]["category"]] == desired_category 
             for idx in top_results
         ) - true_positives

Thanks in advance for your help.

1 Like
2

I believe this is an error in the code. To calculate false negatives, we need to find documents that have the correct category, but are not included in the top k. So the expression should not sum over top_results, in my opinion, but over all documents, and then subtract the true positives.

1 Like
3

Thank you! That clears it up.