Attention mask and token id

Short Course Q&A Finetuning Large Language Models
1

The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input’s attention_mask to obtain reliable results. Setting pad_token_id to eos_token_id:0 for open-end generation.

Whenever I am running the inference, I am getting above suggestions. Can anyone explain what it is and how to implement them?

2

Hey there, i was also facing the same error and after tinkering with the tokenizer for a bit i found out that inputs = tokenizer.encode(prompt) returns just the input_ids but not the attention mask, Whereas inputs = tokenizer(prompt) returns both the input_ids and the attention mask.

So if you replace this code

inputs = tokenizer.encode(prompt)
output = model.generate(inputs)

With this

inputs = tokenizer(prompt)
output = model.generate(**inputs )

The error warning goes away. Hope that helps :blush:.