Problem with remove_stopwords( )

[Mario_Mesas]

Hi all,

I am having some trouble understanding the difference between using the word_index argument of the Tokenizer and the split( ) built-in function in the remove_stopwords( ) function. Both are yielding the same result in the test meant for the function, but the problem comes in the parse_data_from_file( ) function usage.

1. When using the Tokenizer argument in the remove_stopwords( ):

My function looks like this:

The outcome of parse_data_from_file( ) is this:

2. When using sentence.split( ):

My remove_stopwords( ) function is as follows:

The output of parse_data_from_file( ) is shown, as expected, like so:

Does anyone know why the tokenizer option is not working? Thank you in advance.

[balaji.ambresh]

Tokenizer uses a much more sophisticated mechanism to create tokens from sentence than simply split by space. For this assignment, please use space to create tokens.

Please remove code from your posts. It’s okay to leave outputs and stacktrace though.