End of sentence token - visualization issue

Course Q&A Natural Language Processing NLP with Probabilistic Models
1

Hello,

In the lecture video, it says that we want that the sum of all sentences of lengths = 1. I am not getting a feel for this.

Could you please explain ?

End_of_sentence_token
End_of_sentence_token1873×797 67.5 KB

Thanks
Ani

2

Hi,
It’s the sum of the probability of all sentences of length = 1