Hi All, I am not able to understand what is the role of a masked Token in Decoder Only Model? a. Is Masked Token Prediction during Inference Time where the model is already learnt and it uses the learnt weights to predict the masked Token ? b. If not a, then is it a part of pretraining process…

Predicting Next Set of Tokens in Decoder Model

saifkhanengr August 10, 2023, 5:13am 8

Please check this reply from @Juan_Olano where he explained how decoder only and encoder only models works.

1 Like

Topic		Replies	Views
Transformer decoder architecture in course 2 NLP with Attention Models week-module-2	11	765	April 30, 2024
Masked Attention Transformers Sequence Models coursera-platform	6	957	September 27, 2024
Decoder-only Transformer Training/Inference Sequence Models coursera-platform	3	697	June 6, 2023
I don't understand the transformer's decoder Generative AI with Large Language Models week-module-1	2	225	July 24, 2024
Inference with Transformer Decoder Sequence Models ai-discussions , week-module-5 , coursera-platform	1	24	September 16, 2025