Ungraded Lab : BLEU score

In the implementation of average_clipped_precision function in the BLEU score Lab, we have this condition :

It seems to me that we need to delete the “else” since in the original formula, we have a minimum between candidate count and reference count. These is no mention that we need to implement 0 in case where candidate_count < ref_count

image

Hello @Ibrahim_RIDENE

Yes it does look like the code runs fine without else statement, but probably they added that condition for showing there is no precision score or match between candidate and reference sentences.

@lucas.coutinho Can you have a look on this,

it is from Ungraded lab: BLEU Score, the code run perfectly fine without the else statement too.

Regards
DP

When deleting the else statement, the BLEU score calculated by the function will be different from the one calculated based on sacrebleu library :

So :

  • If the else statement is present : we have same result as the sacrebleu, but we are not implementing the same formula defined
  • Is the else statement is absent, we do not have the same result as the sacrebleu, but we are respecting the formula

→ which sounds weird

Hi @Ibrahim_RIDENE

The else part is for if ngram in reference_n_gram_counts and not for the one above it (note different indentation).

Cheers

Hey all,

Thanks for bringing this issue. I will have a look.

Cheers,
Lucas

Hi @lucas.coutinho

As I mentioned above, this should not be an issue, since the else statement deals with a non-existent ngram and assigns 0 when the ngram is not found - a desired behavior since the reference count is 0 and the min should be 0.

Cheers