Hello,
I got an error at creating function “optimize” resulting in wrong costs value as below.
I have passed all exercised before E.6, so the function “propagate” should be correct.
I am stacked to find the cause of seeing the difference between my costs 0.37904644 and expected_cost 0.31057104.
Make sure that the calculations is accumlative as if you copy the result in new variables you may be loss some significant digits, also make sure from your implementation
1 Like
Notice that your cost after 0 iterations is correct, so the bug is not in your cost code. The bug is in either how you compute the gradients or how you apply the gradients to update w and b.
Actually now that I think a bit more, the gradients are computed by propagate, so that narrows it down further: the bug must be in how you apply the gradients to update the parameters.
Hi Paul and Abd, thank you for your response.
Today, I revise my code starting from propagate function, and cleared Exercise6 without errors.
The difference between yesterday’s one and today is db’s formula from Exercise5.
My wrong one was …
db = np.linalg.norm(1/m * np.sum(A-Y, axis=1))
With this even I could pass E.5 since the result of db was same value, but couldn’t E.6.
Thank you.
Interesting. Thanks for sharing this info. So that must say that there is some limitation of the test case for the propagate function. Of course b and db are scalars, so your code is the equivalent of taking the absolute value of db. That is not general, since gradients can be negative.
I will run some similar experiments and if I can understand why the tests pass, it would be worth filing a bug and suggesting that they make the test cases more diverse with at least one case where the correct value of db is negative.
Ok, there are two test cases: one which is just for “show” and that one actually does have a negative value for db. But you have to notice that your printed value is not the same as the “Expected Value”, but it doesn’t actually “throw” with your code.
But then the test case in propagate_test has a positive result for db, so it does not catch your bug.
Thank you Paul for detail explanation.
I actually forgot about this, gradients can be negative.
My mistake was a good reminder to understand correctly about each parameters and functions.