C3_W1 - understanding Calculating perplexity lab

[Deepti_Prasad]

hi @Omer_Tzuk

Here is a beautiful explanation about perplexity by @jyadav202

Lab : Calculating perplexity

It seems like the following equation for perplexity for Bigram model is doing per-sentence instead of per-word perplexity: PP(W) = \sqrt[m]{\prod_{i=1}^{m}\prod_{j=1}^{|s_i|} \frac{1}{P(w_j^{(i)}| w_{j-1}^{(i)})}} where m is no. of sentences in test set W. |s_i| is the number of words in sentence i. Finally, w_j^{(i)} is j^{th} word in sentence i. It would make sense to me if the -1/m in the above equation was actually N or \sum_{i=1}^{j}{|s_i|} , ie. sum of all words in all sentences. If …

feel free to ask if you still have any query

so basically it logs probability between the given context and predicted word or sentences(corpus), and gives a log of probabilities and the reshaped targets with the axis dimension, helps to get the text or output based on the model created for word or sentence prediction.