Why create a copy of z before calculating Relu?

Course Q&A Natural Language Processing NLP with Probabilistic Models
1

In C2_W4_lecture_nb_5_word_embeddings_step_by_step notebook, this is the implementation for relu:

def relu(z):
    # BEGIN your code here
    result = z.copy()
    result[result < 0] = 0
    # END your code here
    
    return result

I tested the code by not creating the copy, and the result is the same. So my question is why do we need to create a copy then? What are the considerations here? Any tips? Thanks.

2

Be careful: the output is the same, but without the copy you have modified the original z value that was passed in as an argument. Here’s a thread from the DLS forums that explains why that copy is necessary. Not sure you’ll be able to see it if you are only signed up for NLP.

In case you can’t read that thread, the point is that saying this:

result = z

does not “copy” anything. After that statement result and z are both python object references that point at the same object in memory. If you then modify the contents of result, it also modifies the contents of z. That is why the copy is necessary here.

3

I see. Thanks for the explanation! I’ll take a deeper look.

4

Are you able to follow the link that I gave? That gives a really concrete example to show what the point is here.

5

Yes, thanks for the link!

6

Hi,
I have a question related to your answer.
The variable “z” is local to the method “relu”. Why do we need to bother about modification on “z”?
I would simply replace the method definition as

def relu(z):
z[z<0] = 0
return z

7

Maybe I am missing some context here, but if z is passed in to the function as a parameter, it cannot be considered purely local to relu()

The language on whether parameters are passed by reference or value in the Python doc is a little confusing, and behavior depends also on the mutability of the data type passed in, but the tl;dr is be careful to avoid unintended side effects from modifying passed parameter variables.

8

Yes, it is exactly as ai_curious says: z is a reference to an object (numpy array), so it is passed by reference on the procedure call. If you directly modify z in the function without first copying it, you are modifying the global data. Not a good idea. See this thread for some more discussion of this point with a detailed example showing the behavior.

It was also explained with some other links earlier in this thread. What is the point of finding the thread if you don’t actually read it? He asked rhetorically :nerd_face: …

1 Like
9

Oh okay.
By default, Python always uses pass -by-reference values .
So, indeed global z also gets modified. So, one should copy it first.
Thanks.

1 Like