C4_W1rouge1_similarity

Course Q&A Natural Language Processing NLP with Attention Models
1

Hi,

In the function rouge1_similarity, how do we calculate precision and recall?

Thanks.

2

Hi,

I am calculating precision and recall as

# get the precision
precision = overlap/len(ref_counter)

# get the recall
recall = overlap/len(sys_counter)

But I am passing 3 tests and failing 1. How can I resolve this?

Thanks.

3

Hi @vrathi

You should not get the len of ref_counter, but the length of the “system” (the model translation). Note that length of the counter would be smaller if there are duplicate words in a translated sentence.

Similarly, for the recall you should check against the length of the “reference” (the true translation). Also, the same “note” applies as in precision denominator.

Cheers

4

@arvyzukai
What is the “sys_counter”? Do you mean the same as candidate_counter?

Also, it is a frequency table (dictionary), so it should not have any duplicate on keys, right?

5

@arvyzukai

Passed the quick test but failed on the unit test. Have spent seveal hours to debug but no solution. Need help on this issue, Thx.

See screenshot:

NLP C4W1Ex6 ROUGE-1 Score
NLP C4W1Ex6 ROUGE-1 Score902×504 19 KB

6

I figured out the solution.
Basically, in computing precision and recall, you have to account for the “raw” count of candidates and references, not just the “unique” count which would be smaller in case there are duplicates in either candidates or references