Very first input in decoder at bottom

At the bottom of the decoding layer, we input tokens from the output of the decoding layer. However, at the very beginning, what input is provided to the decoding layer?

It depends on the problem. Suppose that you are using it for a translation task, then the very first input to the decoder is the sequence of tokens from the target language (that is a
correct translation of source sentence in a target language). The first token in all the input sequences is a special token.

Thanks. and a small doubt, This first(special) token is already in the trained model. right?

@Ashutosh_Bharti Yes. All the special tokens used for training a model is used during the inference. (One can get all those special tokens from the tokenizer )