this is the output that i am getting for the tokens.
the word index of the labels is 1,2, 3,4, 0 but on output it shows numbers like 781 , 146 etc.
Seems like you are using the sentence tokenizer instead of the label tokenizer. Could you please confirm?
in my code i used label tokenizer only
[code removed - moderator]
Please click my name and message your notebook as an attachment.
In def tokenize_labels, you’ve used tokenizer and not label_tokenizer to integer encode labels.