# Lab : Calculating perplexity

**URL:** <https://community.deeplearning.ai/t/lab-calculating-perplexity/609496>\
**Category:** NLP with Sequence Models\
**Tags:** week-module-1\
**Created:** [April 13, 2024, 7:19pm UTC](https://community.deeplearning.ai/t/lab-calculating-perplexity/609496 "2024-04-13T19:19:21Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ibrahim\_RIDENE](https://avatars.discourse-cdn.com/v4/letter/i/9d8465/32.png) [@Ibrahim\_RIDENE](https://community.deeplearning.ai/u/Ibrahim_RIDENE)\
**Post date:** [April 13, 2024, 7:19pm UTC](https://community.deeplearning.ai/t/lab-calculating-perplexity/609496/1 "2024-04-13T19:19:21Z")

</div>

Hi, The formula of the perplexity in the Lab is not the same as the formula discussed during course 2 of the specialization.  
This is the formula that we have in the second course( same as wikipedia) 😊  
 ![image](https://global.discourse-cdn.com/dlai/original/3X/a/d/adfecd9c5587216d60b5ca8e515521f161ede456.png)

The one in the Lab is  
 ![image](https://global.discourse-cdn.com/dlai/original/3X/d/f/dfb2514ae0dcd3f16942fbf3546ff6e9c9b7978e.png)  
where we have the same number N

---

<div class="post-metadata">

**Author:** ![Deepti\_Prasad](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/deepti_prasad/32/165831_2.png) [@Deepti\_Prasad](https://community.deeplearning.ai/u/Deepti_Prasad)\
**Post date:** [April 13, 2024, 9:02pm UTC](https://community.deeplearning.ai/t/lab-calculating-perplexity/609496/2 "2024-04-13T21:02:41Z")

</div>

Hello @Ibrahim_RIDENE

Kindly share the wikipedia link you are referring remember, so a proper explanation can be provided after reviewing and comparing both lab and wikipedia information as stated by you.

Regards  
DP

---

<div class="post-metadata">

**Author:** ![jyadav202](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/jyadav202/32/162439_2.png) [@jyadav202](https://community.deeplearning.ai/u/jyadav202)\
**Post date:** [April 14, 2024, 5:11am UTC](https://community.deeplearning.ai/t/lab-calculating-perplexity/609496/3 "2024-04-14T05:11:01Z")

</div>

Hi @Ibrahim_RIDENE ,

Both formula’s are correct. The second one is derived from the first in case of bi-gram model, with the condition that all the sentences in the test set gets concatenated.  
 ![image](https://global.discourse-cdn.com/dlai/original/3X/4/8/482107b0272b9bcf7413bc98a22a78faebe3054e.png)  
This is talked about in Language Model Evaluation chapter of Autocomplete at timestamp 5:00

---

<div class="post-metadata">

**Author:** ![Ibrahim\_RIDENE](https://avatars.discourse-cdn.com/v4/letter/i/9d8465/32.png) [@Ibrahim\_RIDENE](https://community.deeplearning.ai/u/Ibrahim_RIDENE)\
**Post date:** [April 14, 2024, 11:42am UTC](https://community.deeplearning.ai/t/lab-calculating-perplexity/609496/4 "2024-04-14T11:42:20Z")

</div>

Hi,  
Please find the link to the wikipedia formula: [Perplexity - Wikipedia](https://en.wikipedia.org/wiki/Perplexity#Perplexity_per_word).

@jyadav202 , I would like to mention that even the formula that you provided is not totally correct following this explanation

- The m of the root is not the same as the m of the first product inside the root. The first one must be the number of the distinct words in the corpus, the second one is the number of sequences in the corpus  
 ![image](https://global.discourse-cdn.com/dlai/original/3X/0/8/08f674abd803f1a66a26a62e86b436eedb892832.png)

---

<div class="post-metadata">

**Author:** ![jyadav202](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/jyadav202/32/162439_2.png) [@jyadav202](https://community.deeplearning.ai/u/jyadav202)\
**Post date:** [April 14, 2024, 4:08pm UTC](https://community.deeplearning.ai/t/lab-calculating-perplexity/609496/5 "2024-04-14T16:08:15Z")

</div>

It seems like the following equation for perplexity for Bigram model is doing per-sentence instead of per-word perplexity:

PP(W) = \sqrt[m]{\prod\_{i=1}^{m}\prod\_{j=1}^{|s\_i|} \frac{1}{P(w\_j^{(i)}| w\_{j-1}^{(i)})}}

where m is no. of sentences in test set W. |s\_i| is the number of words in sentence i. Finally, w\_j^{(i)} is j^{th} word in sentence i.

It would make sense to me if the -1/m in the above equation was actually N or $  
\sum\_{i=1}^{j}{|s\_i|}$ , ie. sum of all words in all sentences.

If so, when I said:

> [@jyadav202](#):
>
> with the condition that all the sentences in the test set gets concatenated.

it would mean:

i = m=1  
|s\_i| = |s\_1| = N

Which gives:

PP(W) = \sqrt[N]{\prod\_{i=1}^{N} \frac{1}{P(w\_i| w\_{i-1})}}

(NOTE: the lab denotes perplexity and probability both as P() which is super confusing.)

Anyway, your original query was about this above equation, which is correct and is derived from cross-entopy of the model for probability distribution P. I will not go into the proof but will direct you to this section 3.3 of this link: [https://web.stanford.edu/~jurafsky/slp3/3.pdf](https://web.stanford.edu/~jurafsky/slp3/3.pdf)

I would also take the suggestion to the content creator of the slides and lab to give clarity to this equation.

Thanks!

---

<div class="post-metadata">

**Author:** ![Ibrahim\_RIDENE](https://avatars.discourse-cdn.com/v4/letter/i/9d8465/32.png) [@Ibrahim\_RIDENE](https://community.deeplearning.ai/u/Ibrahim_RIDENE)\
**Post date:** [April 14, 2024, 5:03pm UTC](https://community.deeplearning.ai/t/lab-calculating-perplexity/609496/6 "2024-04-14T17:03:56Z")

</div>

Thanks for the clarification ! Really appeciate it 😊