Bleu and rouge scores

Course Q&A Natural Language Processing NLP with Attention Models
1

I find this subject very confusing in the lecture slides, and as a result also in the Exam questions, and the programming exercise (UNQ_C8).

First of all, in the lectures there are two versions of BLEU: The first include token duplications in the candidate string, and the second (improved) version does not include duplications. Later on, when you refer to ‘BLEU’ it is not really clear which one of the two options is the “correct” one.

In addition, there is no explanation on how to deal with duplications in the reference string, when calculating ROUGE score. Also, there is not “reading” section on any of them.
I feel that there is a place for improvement and consistency in the way this subject is being presented.

2

I agree, it is very confusing. Especially in programming assignment where we have to calculate ROUGE score and for some reason in the assignment ROUGE score is calculated with F1 score formula

6

I agree. I didn’t understand why they use F1 score in the ROUGE function