Week 4 Excercise 10 Programming Assignment # 1

In the update parameters section, I found something that doesn’t make sense to me intuitively coming from a python background. This is with regards to the weight update code, which trivial enough is simply an assignment, right?
So if I had to assign w to w - learning_rate * dw I’d do this:

w -= learning_rate*dw

But this leads to failed tests!
I had used a similar weight update rule:

 parameters["W" + str(l+1)] -= learning_rate*grads["dW" + str(l+1)]

I felt like this was cleaner than the alternative, alas, this leads to failed tests.

But, when I do the dirtier and uglier assignment i.e this:

  parameters["W" + str(l+1)] =  parameters["W" + str(l+1)] - learning_rate* grads["dW" + str(l+1)]

It works!
Can someone please point out the very obvious flaw in my understanding here?

1 Like

It has to do with the fact that objects are passed by reference in python. So if you use the “in place” operators like -=, you are actually modifying the global definitions of the arguments that are passed in, unless you take other action to create private copies. In other words, the two methods generate the same answer, but the way they manage memory is very different. Here’s a thread which discusses this in more detail. That thread discusses the case in Week 3, but the issue is the same here in Week 4. If the global values are passed in to more than one test, the subsequent tests all fail because the input values have been changed, unless you take action to avoid that side effect as described on that thread I linked.


Glad I came across this mistake of mine! And it makes perfect sense to me now!
When I used the in-place update operator, the W was actually being assigned to the object related to parameters[‘W1’] , and that meant that I was actually updating the parameters dictionary!

If anyone stumbles on to this thread, I’d highly recommend going through the thread linked by @paulinpaloalto as well.

Thanks guys this was really helpful!

this is immensely helpful in solving the same problem I struggled for an evening. Thank you!