Failed testcases for w3_unittest.test_preprocess_data(preprocess_data)

Course Q&A Natural Language Processing NLP with Probabilistic Models
1

I have 3 failed testcases for # Test your function w3_unittest.test_preprocess_data(preprocess_data). May I know why ?

(code snippets removed)

1 Like
2

@user342 Your post is 8 days old. I don’t know if your issue is resolved. If not then also pass unknown_token while calling replace_oov_words_by_unk. Probably the test procedure is calling preprocess_data with a different unknown_token.

2 Likes
3

I have used your suggestion as below, but it is not helping with the test procedure.

(Code snippet removed)

4

@user342 Just use unknown_token in place of unknown_token=""

3 Likes
6

I am facing a similar issue, I have tried passing unknown_token while calling replace_oov_words_by_unk Please help me.

vocabulary = get_words_with_nplus_frequency(train_data,count_threshold)

# For the train data, replace less common words with "<unk>"
train_data_replaced = replace_oov_words_by_unk(train_data,vocabulary,unknown_token="")

# For the test data, replace less common words with "<unk>"
test_data_replaced = replace_oov_words_by_unk(test_data,vocabulary,unknown_token="")
7

Replace unknown_token="" with either unknown_token or unknown_token=unknown_token.