Gradient_check_test in the notebook passes on wrong output

Hello,

It looks like gradient_check_test() does not currently check anything and should be fixed. First I found it when I made a mistake in gradient_check() code and next cell gave me an error message from inside gradient_check function and all tests pass from the test function.

To confirm it, I fed stub function to gradient_check_test function and it also passed:

Copyable code:

def gradient_check_stub(x, theta, epsilon=1e-7, print_msg=False):
    return 1.0

gradient_check_test(gradient_check_stub)

gradient_check_test is in /home/jovyan/work/release/W1A3/public_tests.py module and its source code on July, 7, 2021 is:

def gradient_check_test(target):
    x, theta = 2, 4
    expected_output = 2.919335883291695e-10
    test_cases = [
        {
            "name": "equation_output_check",
            "input": [x, theta],
            "expected": expected_output,
            "error": "Wrong output"
        } 
    ]
    
    single_test(test_cases, target)

Actually, output check in the notebook does not work in other tests too because simple_test() checks output for (expected <= actual) while must check for approximate equality:

def single_test(test_cases, target):
    success = 0
    for test_case in test_cases:
        target_output = target(*test_case['input'])
        try:
            if test_case['name'] == "datatype_check":
                assert isinstance(target_output,
                                  type(test_case["expected"]))
                success += 1
            if test_case['name'] == "equation_output_check":
                assert np.less_equal(test_case["expected"],
                                   target_output)  # << should be np.allclose there
                success += 1
            if test_case['name'] == "shape_check":
                assert test_case['expected'].shape == target(*test_case['input']).shape
                success += 1
            if test_case['name'] == "multiple_equation_output_check":
                assert np.any(np.isclose(target_output, test_case['expected']))
                success += 1
        except:
            print("Error: " + test_case['error'])
            
    if success == len(test_cases):
        print("\033[92m All tests passed.")
    else:
        print('\033[92m', success," Tests passed")
        print('\033[91m', len(test_cases) - success, " Tests failed")
        raise AssertionError("Not all tests were passed for {}. Check your equations and avoid using global variables inside the function.".format(target.__name__))
1 Like

This assert also does not look right:

if test_case['name'] == "multiple_equation_output_check":
   assert np.any(np.isclose(target_output, test_case['expected']))

Should not it be np.all() instead of np.any()?

1 Like

Hi, @ViktorDremov.

Thank you for taking the time to look into the problem.

This issue has already been reported. I will add your comments to it.

Hope you’re enjoying the course :slight_smile: