Hi Everyone!
so far we have learnt that generative ai was trained on large data to predict the next word.
but when we give it a prompt or context to act like a sentiment analyzer. how is it able to do this? how does it know the meaning behind good or bad as words. and how is it able to review the restaurant reviews?
Hi @Deeksha_D ,
In your question, you already have part of the answer. See the words “trained”, “large data”, “context”, “act like a sentiment analyzer”.
During the pretraining, the model sees billions of word sequences and learns to predict missing or next tokens. Over many layers, the mechanism of the transformer’s attention composes the token vectors into contextualized representations that capture syntax, semantics.
Predicting the next word forces the model to encode information about topics, sentiment, and relationships between words. Then, the model, implicitly, learns signals such as positive vs negative adjectives, negative patterns, intensifiers, and common phrases.
If you want to go deeper on this stuff, you can take a look at the course How Transformers LLMs Works
Keep learning!
2 Likes
I agree with Carlos. What I understand is that LLMs don’t comprehend sentiment the way humans do. During training on massive amounts of text, they learned the statistical patterns associated with positive and negative words, intensifiers and common phrases. There is no real understanding, just pattern recognition at scale. As a consequence, when someone uses irony or sarcasm, the model struggles more because those patterns are more mbiguous or less frequent in the training data (but not impossible).
3 Likes