W4_A1_subtlety in updating parameters

Course Q&A Deep Learning Specialization Neural Networks and Deep Learning
1

Hi all,

There is a subtlety in updating parameters in the first assignment of week 4, so I think it is good to post it, in case anyone else is also confused by it.

Specifically, let’s look at the following two possible implementations

# first implementation
parameters["W" + str(l+1)] = parameters["W" + str(l+1)] - learning_rate * grads["dW" + str(l+1)] 

# second implementation
parameters["W" + str(l+1)] -= learning_rate * grads["dW" + str(l+1)]

The first can pass the test while the second cannot, although they seems to give the same output for W1, b1, W2, b2. This confused me for a while (~hour).

It turns out that if we print out the id of parameters["W" + str(l+1)], we would find the id is changed in the first implementation, while it is not changed in the second way. And if we look at the file public_tests.py line 481-line499, there are three test cases. So in the second implementation, the value of parameters are changed after the first test case, which brings the error.

Hope this helps!

Yuhan

2

Yes, thanks for noticing and figuring out the underlying reason. The semantics of passing objects to functions in python can get you in trouble. You can use copy or deepcopy on the parameters dictionary to avoid this problem and still be able to use the “in-place” operators.

Here’s a thread which talks more about this case and points out some other issues with passing objects if you read the later posts on that thread.