I can’t find any mentioning of the word_index attribute in the docs for the Tokenizer API on the TensorFlow website. I would appreciate any direction.
word_index refers the mapping from a token in the tokenizer vocabulary to the integer assigned to it.
print(tokenizer.word_index) to see the output.
Do file a ticket on the tensorflow repository to add documentation about
index_word which provides the mapping from token id to the token if you want to improve the documentation.