Ungraded Lab : BLEU score

Course Q&A Natural Language Processing NLP with Attention Models
1

In the implementation of average_clipped_precision function in the BLEU score Lab, we have this condition :

image
image754×621 38 KB

It seems to me that we need to delete the “else” since in the original formula, we have a minimum between candidate count and reference count. These is no mention that we need to implement 0 in case where candidate_count < ref_count

image

3

Hello @Ibrahim_RIDENE

Yes it does look like the code runs fine without else statement, but probably they added that condition for showing there is no precision score or match between candidate and reference sentences.

@lucas.coutinho Can you have a look on this,

it is from Ungraded lab: BLEU Score, the code run perfectly fine without the else statement too.

Regards
DP

4

When deleting the else statement, the BLEU score calculated by the function will be different from the one calculated based on sacrebleu library :

image
image838×378 26 KB

So :

  • If the else statement is present : we have same result as the sacrebleu, but we are not implementing the same formula defined
  • Is the else statement is absent, we do not have the same result as the sacrebleu, but we are respecting the formula

→ which sounds weird

5

Hi @Ibrahim_RIDENE

The else part is for if ngram in reference_n_gram_counts and not for the one above it (note different indentation).

Cheers

6

Hey all,

Thanks for bringing this issue. I will have a look.

Cheers,
Lucas

7

Hi @lucas.coutinho

As I mentioned above, this should not be an issue, since the else statement deals with a non-existent ngram and assigns 0 when the ngram is not found - a desired behavior since the reference count is 0 and the min should be 0.

Cheers