Replace token with <unk> test cases failing

Course Q&A Natural Language Processing NLP with Probabilistic Models
1

Hi,
I am stuck with this assignment where 3 test cases were failing for the preprocess_data function. I really don’t understand where it is going wrong. I also tried to get the original state of the assignment to rework on this but no luck. Can you please help me here?

A few test cases were failing in Exercise 07 where I am trying to "Replace tokens that appear less than count_threshold times by “” "

Wrong number of unknown tokens in the test_data_replaced list. Check the unknown token value and how you are using it.
	 Expected: 4 
	Got: 0.
Wrong number of unknown tokens in the train_data_replaced list. Check the unknown token value and how you are using it.
	 Expected: 10 
	Got: 0.
Wrong number of unknown tokens in the test_data_replaced list. Check the unknown token value and how you are using it.
	 Expected: 7 
	Got: 0.

Hope this information helps and I can share more details if needed.

Thanks in advance.

1 Like
2

Hey @ashwanth467,
Welcome to the community. In this exercise, the words that appear less than count_threshold times are to be replaced by unknown_token, which is being passed to the function as a parameter, and not by an empty string “”. I hope this helps.

Cheers,
Elemento

3

Thanks for pointing out. I am always passing the unknown_token="" which caused the error