# TF Dev specialization, Course-3,wk-1, fit\_tokenizer():

**URL:** <https://community.deeplearning.ai/t/tf-dev-specialization-course-3-wk-1-fit-tokenizer/277361>\
**Category:** Natural Language Processing in TensorFlow\
**Tags:** week-module-1\
**Created:** [February 20, 2023, 11:16am UTC](https://community.deeplearning.ai/t/tf-dev-specialization-course-3-wk-1-fit-tokenizer/277361 "2023-02-20T11:16:47Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![ydsk1234](https://avatars.discourse-cdn.com/v4/letter/y/e8c25b/32.png) [@ydsk1234](https://community.deeplearning.ai/u/ydsk1234)\
**Post date:** [February 20, 2023, 11:16am UTC](https://community.deeplearning.ai/t/tf-dev-specialization-course-3-wk-1-fit-tokenizer/277361/1 "2023-02-20T11:16:47Z")

</div>

Hello,

the graded cell for “def fit\_tokenizer(sentences)” is simple but it gives a slightly different output than expected (29731 words instead of the expected 29714):

 ![image](https://global.discourse-cdn.com/dlai/original/3X/0/d/0d6d700bf16ac00f13577391a45d8a381954d51c.png)

my code has just 2 lines, tokenizer constructor(with oov\_token=), followed by its fit\_on\_texts(sentences).

What could have possibly gone wrong?

---

<div class="post-metadata">

**Author:** ![ai\_curious](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/ai_curious/32/128003_2.png) [@ai\_curious](https://community.deeplearning.ai/u/ai_curious)\
**Post date:** [February 20, 2023, 11:50am UTC](https://community.deeplearning.ai/t/tf-dev-specialization-course-3-wk-1-fit-tokenizer/277361/2 "2023-02-20T11:50:22Z")

</div>

Did you know there is a search function on this platform?

 ![image](https://global.discourse-cdn.com/dlai/original/3X/e/2/e2c42b4fbe33a0d3bc8c569b2b527063cdde4066.png)

---

<div class="post-metadata">

**Author:** ![ydsk1234](https://avatars.discourse-cdn.com/v4/letter/y/e8c25b/32.png) [@ydsk1234](https://community.deeplearning.ai/u/ydsk1234)\
**Post date:** [February 21, 2023, 10:39am UTC](https://community.deeplearning.ai/t/tf-dev-specialization-course-3-wk-1-fit-tokenizer/277361/3 "2023-02-21T10:39:10Z")

</div>

I couldn’t get any hints using the ‘search’ function and that’s the reason for my post.

The assignment is graded and it passed successfully with a full score (20 out of 20 for the graded cell).

Still I want to understand why there’s a difference in the count of words (I am not using num\_words parameter in the Tokenizer constructor call as we don’t know what number to use)

Any hints please?

---

<div class="post-metadata">

**Author:** ![ai\_curious](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/ai_curious/32/128003_2.png) [@ai\_curious](https://community.deeplearning.ai/u/ai_curious)\
**Post date:** [February 21, 2023, 11:42am UTC](https://community.deeplearning.ai/t/tf-dev-specialization-course-3-wk-1-fit-tokenizer/277361/4 "2023-02-21T11:42:17Z")

</div>

> [@ydsk1234](#):
>
> couldn’t get any hints using the ‘search’ functio

Take a closer look at that screen capture. It’s the exact same question about the exact same exercise. The mentor replied with suggestions, the original poster wrote back that it solved the problem. Are you saying that it’s a coincidence that the value returned, 29731, is the same as that post, but your problem in the code is different? That just seems improbable.