# C3W1\_Assignment fit\_tokenizer() 00V problem

**URL:** <https://community.deeplearning.ai/t/c3w1-assignment-fit-tokenizer-00v-problem/518485>\
**Category:** Natural Language Processing in TensorFlow\
**Tags:** week-module-1\
**Created:** [December 14, 2023, 8:24am UTC](https://community.deeplearning.ai/t/c3w1-assignment-fit-tokenizer-00v-problem/518485 "2023-12-14T08:24:56Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Alberto\_Zacchini](https://avatars.discourse-cdn.com/v4/letter/a/f05b48/32.png) [@Alberto\_Zacchini](https://community.deeplearning.ai/u/Alberto_Zacchini)\
**Post date:** [December 14, 2023, 8:24am UTC](https://community.deeplearning.ai/t/c3w1-assignment-fit-tokenizer-00v-problem/518485/1 "2023-12-14T08:24:56Z")

</div>

Hi guys!  
I have a problem with the 00V token detection. When i fit the tokenizer and i sequence the senteces, i get the correct number of words in the vocabulary but also i get “ token NOT included in vocabulary”.

I initialized the tokenizer as i did during the class in the same way, but still dont get the 00V included in the vocabulary.

Any tips?

My lab: ldzjcyddmhcu

---

<div class="post-metadata">

**Author:** ![balaji.ambresh](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/balaji.ambresh/32/128697_2.png) [@balaji.ambresh](https://community.deeplearning.ai/u/balaji.ambresh)\
**Post date:** [December 14, 2023, 8:38am UTC](https://community.deeplearning.ai/t/c3w1-assignment-fit-tokenizer-00v-problem/518485/2 "2023-12-14T08:38:16Z")

</div>

Please click my name and message your notebook as an attachment.

---

<div class="post-metadata">

**Author:** ![balaji.ambresh](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/balaji.ambresh/32/128697_2.png) [@balaji.ambresh](https://community.deeplearning.ai/u/balaji.ambresh)\
**Post date:** [December 14, 2023, 4:15pm UTC](https://community.deeplearning.ai/t/c3w1-assignment-fit-tokenizer-00v-problem/518485/3 "2023-12-14T16:15:58Z")

</div>

Please find the feedback below:

1. The out of vocabulary token to use is `<OOV>` and not `<00V>` (not zeros but upper case alphabet i.e. O)
2. See the function `fit_tokenizer` to understand how word to index mapping is referenced from inside the tokenizer instance in the test code. This will help fix the mistakes in `tokenize_labels`