Hi all,
Correct me if my understanding is wrong, in NLP the NN model predicts the next word when the context is given right. The NN is trained on Large corpus of data sets and during initial predictions we use teacher forcing method and when the model is ready ., the context is given lets say " I want a glass of orange {prediction word} " and the word is " Juice " is predicted. Am I right.
Hi @Krish_code
Yes. In NNs like language models predict the next word based on a given context (during training, a teacher forcing is used to have a effective training process). Once trained, the model predicts the next word based on the input context.
In your example, the context is āI want a glass of orangeā and the model would predict ājuiceā as the next word if it has learned the relation between āorangeā and ājuiceā from the training data.
Hope it helps! Feel free to ask if you need further assistance.
1 Like
Thank you for responding to my message. further more I am diving into Transformers and LLM architectures., need someone to clear my doubts. Will definitely approach you in near future. Thanking you once again.
Youāre welcome! Sure, feel free to reach out whenever you need help.
@Krish_code just worth noting, as I found this confusing when first studying, but your āteacher forcingā is your one of off shift. Or it is always one step ahead.
Also if you are really serious about LLMs you should check this: Neural Networks: Zero To Hero
I mean NLP course is great, it fills in a lot of gaps, so I see them as compliments.