Are encoder only architectures still used for sentiment analysis?

In the class from week 1 the teacher said that the use of encoder only architectures is less common these days, but by adding additional layers you can train them to perform sentiment analysis. Does this mean that even though you can use them for sentiment analysis, there are better ways now?

Thank you for the help.
best,
David

1 Like

Encoder only models are very good at extracting features. If you add a final layer that produces a binary result (good/bad, hate, no-hate, etc) or a softmax that produces multiple results, or a regression to predict a score, you can create a very good sentiment analysis.

Now, the new, very large, models of the likes of GPT and Claude, which are decoders only, are great also at sentiment analysis, so many times people may use them. But again, we are talking about the big models. A small decoder may not be as good.

If you want a local solution, I would say that an encoder finetuned and with a head at the end to return the type of result you expect in your sentiment analysis, is a great solution.