Course 1 Week 2 not 100 despite all tests being passed

For the first graded assignment in week 1, https://www.coursera.org/learn/neural-networks-deep-learning/programming/isoAV/python-basics-with-numpy/lab I am passing all tests, but only see an 88.

I’ve made sure my code is between the comments provided, and is the number of lines requested in the problem statement. What might be the cause here?

This is the grader output:

[ValidateApp | INFO] Validating ‘/home/jovyan/work/submitted/courseraLearner/W2A1/Python_Basics_with_Numpy.ipynb’
[ValidateApp | INFO] Executing notebook with kernel: python3
Tests failed on 1 cell(s)! These tests could be hidden. Please check your submission.

The tests in the notebook are just one set of tests. Just passing those tests does not prove your code is completely correct. The grader uses different tests. One type of error that can cause this syndrome is “hard-coding” things, like referencing global variables directly from the context of your functions. Meaning that the evidence suggests that your code is not general in some way: it passes one test and fails another.

But maybe the best thing is for me to check your code. We can’t do that directly here, but I’ll send you a DM (Direct Message) about how to do that.

@lukestiles also, not to be ‘clever’ (or have @paulinpaloalto get mad at me :grin:), but if you poke around a bit, even from just the results of the unit tests (though you can go deeper…). You can kind of ascertain the results the tests expect-- Or, IMHO, reach that point and you are a real programmer.

I mean of course you still have to get your code right, but you are also really debugging at that point.

Sure, you could look at what the unit tests do and then create a new test for yourself.

I looked at Luke’s notebook in a DM thread and the error was exactly what I suggested earlier: one of the functions directly referenced the global variable that happened to get passed as a parameter by the test case. It works in the notebook, but it’s a bug. :nerd_face:

1 Like

The mismatch between the grade and the tests was confusing. Probably best to understand the tests as “You will pass” and the grade as the nuanced view of what sort of passing you will do.

FWIW I made the update based on @paulinpaloalto 's feedback and all is well.

2 Likes

I never went as ‘deep’ as I suggested, but through the course I did find it useful just to look at least only at what is in the notebook, or what the tests are expecting.

I found it helpful to suggest at times what I was missing.

1 Like

@lukestiles
Yeah no, Paul is good :grin:.

I am just ‘naughty’.

Notebooks are great BECAUSE they aren’t that strict as an IDE. The tests run on submission are stricter than the ones run in the notebook. Debugging would probably require clearing the kernel after running a cell to ensure global variables aren’t being used. But that sounds like a lot of work. :slight_smile:

There is a more straightforward way to understand this: just passing one test is not enough to prove that your code is fully correct. With only one test, there are too many ways you can write code that passes, but is not “general”. Meaning that it will fail with a different test case. The example you have just worked through is a clear illustration of how that kind of thing can happen.

In some later parts of the course, they sometimes have more than one test in the notebook for a given function to try to mitigate this type of confusion. But the same rule will apply in general throughout the courses here: just passing the tests in the notebook is not the end of the story.

1 Like

@lukestiles well, do consider in a real IDE, in real debugging you can put in stop points and also see the present state of all your variables, rather than having to do ‘print’ statements. And also you can ‘step through’ a single line at a time (I guess Colab does have stepping, at least).

I’m not sure that is even possible in Colab, so perhaps I’d hold that thought.

Exactly.

I would not say the submit tests are stricter. They are merely different tests than those provided in the notebook.

1 Like