"Percent Improvement" in Lab 2 Walkthrough Video


I was watching the lab 2 walkthrough video (haven’t started the lab yet), and at around 11:29 / 17:26 into the video, we see the percent improvements as “18.82%, 10.43%, 13.70%”, etc.

I wanted to ask about this, since for example, if we look at rouge1 (screenshot posted below), wouldn’t it be:

( (0.4216129 - 0.2334158) / (0.2334158) ) * 100 = 80.63% improvement in rouge score?

The rouge1 score goes from 0.2334158 in the original model to 0.4216129 in the instruct full fine-tuned model.

I’m using the percent increase / decrease formula, since 18.82% seems like a small percent increase, when the rouge value nearly doubles!

This is because they are calculating absolute percentage vs. your formula which is for relative percentage, see below: