C3_W1 - understanding Calculating perplexity lab

[Omer_Tzuk]

I am trying to understand how the calculations presented in this lab (link) represent the perplexity.

Specifically, I don’t grasp how “By calculating the product of the predictions and the reshaped targets and summing across the last dimension, the total log probability of each observed element within the sequences can be computed:”

log_p = np.sum(predictions * reshaped_targets, axis= -1)

I would appreciate any clarification.

[Deepti_Prasad]

hi @Omer_Tzuk

Here is a beautiful explanation about perplexity by @jyadav202

Lab : Calculating perplexity

It seems like the following equation for perplexity for Bigram model is doing per-sentence instead of per-word perplexity: PP(W) = \sqrt[m]{\prod_{i=1}^{m}\prod_{j=1}^{|s_i|} \frac{1}{P(w_j^{(i)}| w_{j-1}^{(i)})}} where m is no. of sentences in test set W. |s_i| is the number of words in sentence i. Finally, w_j^{(i)} is j^{th} word in sentence i. It would make sense to me if the -1/m in the above equation was actually N or \sum_{i=1}^{j}{|s_i|} , ie. sum of all words in all sentences. If …

feel free to ask if you still have any query

so basically it logs probability between the given context and predicted word or sentences(corpus), and gives a log of probabilities and the reshaped targets with the axis dimension, helps to get the text or output based on the model created for word or sentence prediction.