Week 3 - Timeout causing test failure during submission

Hi there,

All the tests pertaining to the Week 3 programming assignment pass when I manually run them. But when I submit the assignment, this error occurs (it appears in the grader output):

[ValidateApp | INFO] Validating ‘/home/jovyan/work/submitted/courseraLearner/W3A1/Planar_data_classification_with_one_hidden_layer.ipynb’
[ValidateApp | INFO] Executing notebook with kernel: python3
[ValidateApp | ERROR] Timeout waiting for execute reply (30s).
[ValidateApp | ERROR] Interrupting kernel
Tests failed on 1 cell(s)! These tests could be hidden. Please check your submission.

The error doesn’t tell me which cell I need to focus on for troubleshooting.

Can you please advise me on how to approach this issue?

Thank you.

Hi @vng, I suggest to retry to check whether it was a intermittent issue on server side. If it still throws an issue, please make sure you are not using any hardcoded values or referencing global variables. This could lead to the shown error.

1 Like

The other point to make is that the “timeout” messages are normal. Here’s what the grader output looks like from a successful run:

[ValidateApp | INFO] Validating '/home/jovyan/work/submitted/courseraLearner/W3A1/Planar_data_classification_with_one_hidden_layer.ipynb'
[ValidateApp | INFO] Executing notebook with kernel: python3
[ValidateApp | ERROR] Timeout waiting for execute reply (30s).
[ValidateApp | ERROR] Interrupting kernel
Success! Your notebook passes all the tests.

Sorry that the grader does not give more useful feedback about which cell is actually causing your problems. I’ve tried making various mistakes and in all the cases I tried, it gave me more specific feedback. E.g. here’s what it looks like when I purposely broke the initialize routine:

[ValidateApp | INFO] Validating '/home/jovyan/work/submitted/courseraLearner/W3A1/Planar_data_classification_with_one_hidden_layer.ipynb'
[ValidateApp | INFO] Executing notebook with kernel: python3
[ValidateApp | ERROR] Timeout waiting for execute reply (30s).
[ValidateApp | ERROR] Interrupting kernel
Tests failed on 2 cell(s)! These tests could be hidden. Please check your submission.
==========================================================================================
The following cell failed:

    n_x, n_h, n_y = initialize_parameters_test_case()
    parameters = initialize_parameters(n_x, n_h, n_y)
    
    print("W1 = " + str(parameters["W1"]))
    print("b1 = " + str(parameters["b1"]))
    print("W2 = " + str(parameters["W2"]))
    print("b2 = " + str(parameters["b2"]))
    
    initialize_parameters_test(initialize_parameters)

The error was:

    ---------------------------------------------------------------------------
    AssertionError                            Traceback (most recent call last)
    <ipython-input-10-2a1744f50b5b> in <module>
          7 print("b2 = " + str(parameters["b2"]))
          8 
    ----> 9 initialize_parameters_test(initialize_parameters)
    
    ~/work/submitted/courseraLearner/W3A1/public_tests.py in initialize_parameters_test...
         65     ]
         66 
    ---> 67     multiple_test(test_cases, target)
         68 
         69 def forward_propagation_test(target):
    
    ~/work/submitted/courseraLearner/W3A1/test_utils.py in multiple_test(test_cases, ta...
        162         print('\033[91m', len(test_cases) - success, " Tests failed")
        163         raise AssertionError(
    --> 164             "Not all tests were passed for {}. Check your equations and avo...
    
    AssertionError: Not all tests were passed for initialize_parameters. Check your equ...


==========================================================================================
The following cell failed:

    t_X, t_Y = nn_model_test_case()
    parameters = nn_model(t_X, t_Y, 4, num_iterations=10000, print_cost=True)
    
    print("W1 = " + str(parameters["W1"]))
    print("b1 = " + str(parameters["b1"]))
    print("W2 = " + str(parameters["W2"]))
    print("b2 = " + str(parameters["b2"]))
    
    nn_model_test(nn_model)

The error was:

    ---------------------------------------------------------------------------
    AssertionError                            Traceback (most recent call last)
    <ipython-input-26-6d3d4c22ad68> in <module>
          7 print("b2 = " + str(parameters["b2"]))
          8 
    ----> 9 nn_model_test(nn_model)
    
    ~/work/submitted/courseraLearner/W3A1/public_tests.py in nn_model_test(target)
        311     ]
        312 
    --> 313     multiple_test(test_cases, target)
        314 
        315 def predict_test(target):
    
    ~/work/submitted/courseraLearner/W3A1/test_utils.py in multiple_test(test_cases, ta...
        162         print('\033[91m', len(test_cases) - success, " Tests failed")
        163         raise AssertionError(
    --> 164             "Not all tests were passed for {}. Check your equations and avo...
    
    AssertionError: Not all tests were passed for nn_model. Check your equations and av...
1 Like

Thank you for your suggestion. I have used copy.deepcopy where advised but the issue still exists.

Thank you for letting me know that timeouts are not linked with test case failures. Hoping that I can figure out the reason for the failure in my case.

Also, thank you so much for showing me how the grader behaves when it detects failed tests.

I am getting similar issue and I have verified that I am not using global variables, my code changes are within YOUR CODE STARTS HERE to YOUR CODE ENDS HERE areas, locally all cells pass.
How to find the offending cell?

[ValidateApp | INFO] Validating ‘/home/jovyan/work/submitted/courseraLearner/W3A1/Planar_data_classification_with_one_hidden_layer.ipynb’
[ValidateApp | INFO] Executing notebook with kernel: python3
[ValidateApp | ERROR] Timeout waiting for execute reply (30s).
[ValidateApp | ERROR] Interrupting kernel
Tests failed on 1 cell(s)! These tests could be hidden. Please check your submission.

Ok, we have an answer on Raman’s problem. It turns out that the implementation of the predict function resulted in a 1D array like this:

Predictions: [ True False  True]
All tests passed!

The problem is that this passes the unit tests, but does not pass the grader. That is a bug in the unit tests, which I will file. Here is the “Expected Output” for that test case:

Predictions: [[ True False  True]]

See the difference?

I have the same output as you, but my grade is still 0:

It is odd that the grader output says “Your notebook passes all test tests” and then gives 0 points. Note that the “Interrupting kernel” message is normal (see my earlier post on this thread).

Please try submitting your notebook again. Just to be on the safe side, do the following procedure:

  1. Kernel → Restart and Clear Output
  2. Save
  3. Submit Assignment

And then let us know if you still get 0 points.

1 Like