Course 1 Week 3 BackPropogation Error: Wrong value of dw1

Course Q&A Deep Learning Specialization Neural Networks and Deep Learning
1

Hello! I seem too be having some issues in one of the test cases for backpropagation.

dW1 = [[ 0.00301023 -0.00747267]
 [ 0.00257968 -0.00641288]
 [-0.00156892  0.003893  ]
 [-0.00652037  0.01618243]]
db1 = [[ 0.00176201]
 [ 0.00150995]
 [-0.00091736]
 [-0.00381422]]
dW2 = [[ 0.00078841  0.01765429 -0.00084166 -0.01022527]]
db2 = [[-0.16655712]]
---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
<ipython-input-16-a06d396e2b09> in <module>
      7 print ("db2 = "+ str(grads["db2"]))
      8 
----> 9 backward_propagation_test(backward_propagation)

~/work/release/W3A1/public_tests.py in backward_propagation_test(target)
    187     assert output["db2"].shape == expected_output["db2"].shape, f"Wrong shape for db2."
    188 
--> 189     assert np.allclose(output["dW1"], expected_output["dW1"]), "Wrong values for dW1"
    190     assert np.allclose(output["db1"], expected_output["db1"]), "Wrong values for db1"
    191     assert np.allclose(output["dW2"], expected_output["dW2"]), "Wrong values for dW2"

AssertionError: Wrong values for dW1

The above values seem to match the expected output. Can you tell where I may have gone wrong

Expected output

dW1 = [[ 0.00301023 -0.00747267]
 [ 0.00257968 -0.00641288]
 [-0.00156892  0.003893  ]
 [-0.00652037  0.01618243]]
db1 = [[ 0.00176201]
 [ 0.00150995]
 [-0.00091736]
 [-0.00381422]]
dW2 = [[ 0.00078841  0.01765429 -0.00084166 -0.01022527]]
db2 = [[-0.16655712]]
All tests passed!
 All tests passed.
2

Welcome to the community.

There are two tests in here.
The first one is to use a test case created by "backward_propagation_test_case(). And, the second one is “backward_propagation_test(backward_propagation)”. It looks like you passed the first one, but failed the 2nd one. So, please check your implementation with the second test.
The test case is in public_tests.py which is in the same directory as a notebook, if you want to see.

3

Hi! I am having a similar problem with the dw1 value. dW2 and db2 match with the expected value but bW1 don’t so I guess might be a problem with the dZ1 or dW1 formula. But I have been checking against the formula and I can’t find the problem. Can somebody help me with this? Thanks!

image
image2004Ă—1590 360 KB

image
image2026Ă—540 120 KB

4

Solved, I had a problem with the dZ1 formula. I try using dZ1 = np.dot(W2.T,dZ2) * (1 - np.power(A1,2)), instead of dZ1 = np.dot(W2.T,dZ2) * np.tanh(1 - np.power(A1,2)) and all the tests are passed. But don’t understand where the tanh of the g’(Z) is represented in the correct one. Can anyone explain me this? Thanks

5

Yes, here is the formula for the derivative of tanh:

tanh'(Z) = (1 - tanh^2(Z))

Since we also have:

A = tanh(Z)

it follows that:

tanh'(Z) = (1 - A^2)

The other potential cause of the problem is that maybe you misinterpreted the formula that you are translating into code. Here it is:

dZ^{[1]} = \left ( W^{[2]T} \cdot dZ^{[2]} \right ) * g^{[1]'}(Z^{[1]})

So there is no g^{[1]} anyplace there. It is only the derivative of g^{[1]}.

4 Likes
6

oh much clear now! Thank you for the explanation @paulinpaloalto

7

Dear sir,
I have a small question about your answer.
here if I use (1 - np.tanh(Z1) ** 2) to replace (1 - A1**2), is it still correct?
I think they shuld be equal except a little accurate issue generated by tanh function.

Thanks,
-Will

8

That is logically correct, but it doesn’t work here because of the way the test cases are written. They just have random values for Z1 and A1 in the input for the test case, so using the formula based on tanh(Z1) fails the test. I’ve filed a bug about this, but it has not been fixed yet.

This issue has come up before. Here’s a thread about it, which also points to this one.