I have encountered an issue that my calculated accuracy for testing set is different than the expected outcome although all the exercises are passed.
Hi @otiliaovo
Did you submit your assignemt to see if it gets full score? You are already achieving a higher accuracy than the expected one!
Hi @Alireza_Saei ,
Yes I have submitted and got 50/50 points. I just wondering how did it happen if with same materials and parameters..
Thanks!
1 Like
You’re welcome!
That’s totally normal! Small differences in accuracy can happen because of random weight initialization or data order, especially with logistic regression using stochastic methods. Since you got 50/50, your code is working as expected!
1 Like
I just checked my version of that notebook and I got the same result:
There are several other cases in which my output does not match the “Expected values” that are shown. Note that the difference between 0.9965 and 0.9950 is not a floating point rounding error: it is much larger than that. If we are using 32 bit floats, then rounding errors are of the order O(10^{-7}) and if we’re using 64 bit floats, they will be O(10^{-15}). You can find more information about that on the Wikipedia page for the IEEE 754 spec.
I think what happened here is that this assignment downloads the input data from the NLTK library and that must have changed. They will need to update the “Expected Values” shown in the notebook to match the changes in the input dataset. I’ll file a bug about that.
3 Likes
Actually this data change affects several of the NLP courses. Here’s a thread about it from February that is specific to this assignment (NLP C1 W1 Logistic Regression).
It looks like they fixed things just enough that the tests pass, even though there are still several inconsistencies between the output values and the expected values.
1 Like
paul perhaps notifying lucas would be a good step, to work on those expected value mismatch as assignment usings nltk data have these issues as nltk was updated and those cause changes.
Regards
DP
1 Like
Thanks, @deepti_prasad. That’s a good idea. Yes, I realized once I did a forum search that you had already filed a bug about some of the instances of issues with the new NLTK data. It looks like they just did minimal fixes so that the tests pass now, but they didn’t notice the “expected value” inconsistencies.
I’ll file a new bug about that …
1 Like