At 6:32 in the video, when the reference sentence is It is cold outside.:
Take, for example, this generated output,
cold, cold, cold, cold.As this generated output contains one of the words from the reference sentence, it will score quite highly even though the same word is repeated multiple times.[emphasis added]
The Rouge-1 precision score will be perfect.
The precision score calculation shown on the whiteboard in the video has 4 as the number of unigram matches, ignoring, as indicated by the transcript, the fact of repetition. But repetitions should not be counted; what should be counted are unique instances. The number of matches should thus be 1, and the precision score should be 1/4. See Google Research’s Python code at
google-research/rouge/rouge_scorer.py at master · google-research/google-research · GitHub
The number of matches, in pseudo-code, is:
for each unigram in reference:
matches += min(reference instances, output instances)