Hi All, I am not able to understand what is the role of a masked Token in Decoder Only Model? a. Is Masked Token Prediction during Inference Time where the model is already learnt and it uses the learnt weights to predict the masked Token ? b. If not a, then is it a part of pretraining process…

Predicting Next Set of Tokens in Decoder Model

saifkhanengr August 10, 2023, 5:13am 8

Please check this reply from @Juan_Olano where he explained how decoder only and encoder only models works.

1 Like

Topic		Replies	Views
Decoder-only architectures for sequence modelling NLP with Attention Models week-2	1	371	August 12, 2023
The tokens that decoder block use Sequence Models week-4	3	209	April 15, 2024
[Week 4] BERT Pre-Training Concepts Sequence Models	1	507	December 12, 2022
Masked Attention Transformers Sequence Models	6	795	September 27, 2024
Token, Encoder an decoder meaning Generative AI with Large Language Models week-1	1	436	July 1, 2023