Compute cost: += vs. =+

The two operations:

a = 1
a += 2
print(a)

a = 1
a = a + 2
print(a)

produce the same answers, as you can demonstrate by running them. They are equivalent in that sense. Where things get more complicated is when the variables involved are python objects like numpy arrays. In that case, the answers are still the same, but the way those two operations manage memory is totally different.

The key point is that a python object is what you would call a “pointer” in a lower level language like c or C++. The variable is just a name that points to a location in memory. Then the distinction between the two methods becomes important: the distinction is that “+=” is an “in-place” operator that actually modifies the object pointed to by the object variable. The second style of assignment allocates a new memory object for the RHS of the assignment statement and then sets the object reference on the LHS to point to the newly allocated object. The main case in which this becomes an issue is when you are passing parameters to functions, because the semantics of passing objects to functions in python is that they are passed “by reference”, not “by value”.

Here’s a little example to demonstrate what I am talking about:

def addOne(input):
    input += 1
    return input

np.random.seed(42)
a = np.random.randint(0, 10, (2,3))
print(f"before:\na = {a}")

b = addOne(a)
print(f"b = {b}")
print(f"after:\na = {a}")

Notice that the addOne function uses the “in place” operator. Now let’s run that and watch what happens:

before:
a = [[6 3 7]
 [4 6 9]]
b = [[ 7  4  8]
 [ 5  7 10]]
after:
a = [[ 7  4  8]
 [ 5  7 10]]

Eeeek! Notice that not only is the return value from the function modified by the +1, so is a, which is the original global variable that was passed in. So that function has a side effect that is probably not a good thing.

To see exactly what is happening, try this in addition:

a[0,2] = 42
print(f"after:\nb = {b}")

Here’s what we get:

after:
b = [[ 7  4 42]
 [ 5  7 10]]

So what has actually happened is that a and b end up both being object references to the same object in memory.

Now here’s an example using the other style of assignment:

def addTwo(input):
    input = input + 2
    return input

np.random.seed(42)
a = np.random.randint(0, 10, (2,3))
print(f"before:\na = {a}")

b = addTwo(a)
print(f"b = {b}")
print(f"after:\na = {a}")

When we run that, here’s what we get:

before:
a = [[6 3 7]
 [4 6 9]]
b = [[ 8  5  9]
 [ 6  8 11]]
after:
a = [[6 3 7]
 [4 6 9]]

Notice that the before and after values of a are the same. So the assignment statement:

input = input + 2

gives the same answer, but manages memory differently. The RHS of that assignment statement is a newly allocated memory object and then the object variable input on the LHS points to the new memory, leaving the global value that was passed in unmodified.

One more point worth making is that there is another way to solve this problem. It’s not a bad idea to get in the habit of “copying” your input parameters when you know that they are object references and the function is going to modify those values. That’s just to be on the safe side and make sure your function doesn’t modify global variables. Here’s a more sophisticated version of the first function:

def addOne(input):
    input = input.copy()
    input += 1
    return input

np.random.seed(42)
a = np.random.randint(0, 10, (2,3))
print(f"before:\na = {a}")

b = addOne(a)
print(f"b = {b}")
print(f"after:\na = {a}")

Running that gives this result:

before:
a = [[6 3 7]
 [4 6 9]]
b = [[ 7  4  8]
 [ 5  7 10]]
after:
a = [[6 3 7]
 [4 6 9]]

So you can see that even though it uses the “in place” += operator on the object, the global object is not modified because we first created a new copy of that object.