Padding question

Course Q&A Deep Learning Specialization Convolutional Neural Networks
1

I tried to practice zero-padding outside of the programing assignment.
As an example, I wrote
A=np.array
([[[ 0, 1, 2],
[ 3, 4, 5],
[ 6, 7, 8]],

   [[ 9, 10, 11],
    [12, 13, 14],
    [15, 16, 17]],

   [[18, 19, 20],
    [21, 22, 23],
    [24, 25, 26]]])

I tried to pad everything with padding=1, so I wrote
B=np.pad(A, ((1,1), (1,1), (0,0)),‘constant’)
But it didn’t work out as expected.
Only
B=np.pad(A, ((0,0), (1,1), (1,1)),‘constant’)
yielded the wanted output (below) Why is that so?
([[[ 0, 0, 0, 0, 0],
[ 0, 0, 1, 2, 0],
[ 0, 3, 4, 5, 0],
[ 0, 6, 7, 8, 0],
[ 0, 0, 0, 0, 0]],

   [[ 0,  0,  0,  0,  0],
    [ 0,  9, 10, 11,  0],
    [ 0, 12, 13, 14,  0],
    [ 0, 15, 16, 17,  0],
    [ 0,  0,  0,  0,  0]],

   [[ 0,  0,  0,  0,  0],
    [ 0, 18, 19, 20,  0],
    [ 0, 21, 22, 23,  0],
    [ 0, 24, 25, 26,  0],
    [ 0,  0,  0,  0,  0]]])
2

This is just a question of understanding how the “print” function interprets arrays with more than 2 dimensions. Here’s a way to understand that:

def testarray(shape):
    (d1,d2,d3) = shape
    A = np.zeros(shape)

    for ii1 in range(d1):
        for ii2 in range(d2):
            for ii3 in range(d3):
                A[ii1,ii2,ii3] = ii1 * 100 + ii2 * 10 + ii3

    return A

So that function creates a “telltale” 3D array and the value in each position gives the indices of that element in order. That is to say A[1,2,3] = 123. Now lets create a 3D array and print it:

A = testarray((3,2,3))
print(f"A = \n{A}")

Notice that I made the middle dimension different sized just to make it a bit more clear what is happening. Here’s what you get:

A = 
[[[  0.   1.   2.]
  [ 10.  11.  12.]]

 [[100. 101. 102.]
  [110. 111. 112.]]

 [[200. 201. 202.]
  [210. 211. 212.]]]

So what you can see is that in the first 2 x 3 array, we have all elements with 0 as the first dimension. In the second, it’s all with index 1 for dimension 1 and so forth. Or to put it another way, what you are seeing laid out there are dimensions 2 and 3. That’s why the padding only “looks right” when you pad dimensions 2 and 3. It’s also correct if you pad dimensions 1 and 2, but the printout just looks funny and you have to work a bit harder to interpret what you are seeing. E.g. one way to do that would be:

print(A[:,:,0])

Or even:

for ii in range(A.shape[2]):
    print(A[:,:,ii])

Note that this is a slightly limited example. We’re really dealing with 4D arrays where the first dimension is “samples”, right? So it is m x h x w x c and you want to pad the h and w dimensions. Try extending the “telltale” function there to support 4D and try some experiments with visualizing the results.

1 Like
3

Let’s extend the experiment a bit:

Apad = np.pad(A, ((1,1), (1,1), (0,0)), mode='constant', constant_values = (0,0))
print(f"Apad.shape = {Apad.shape}")
print(f"Apad = \n{Apad}")

That yields this:

Apad.shape = (5, 4, 3)
Apad = 
[[[  0.   0.   0.]
  [  0.   0.   0.]
  [  0.   0.   0.]
  [  0.   0.   0.]]

 [[  0.   0.   0.]
  [  0.   1.   2.]
  [ 10.  11.  12.]
  [  0.   0.   0.]]

 [[  0.   0.   0.]
  [100. 101. 102.]
  [110. 111. 112.]
  [  0.   0.   0.]]

 [[  0.   0.   0.]
  [200. 201. 202.]
  [210. 211. 212.]
  [  0.   0.   0.]]

 [[  0.   0.   0.]
  [  0.   0.   0.]
  [  0.   0.   0.]
  [  0.   0.   0.]]]

So you can see the first dimension expands from 3 to 5 and the second from 2 to 4. The first 4 x 3 array shown is Apad[0,:,:], so that is all zeros since we padded on dimension 0 and 1. Then we see Apad[1,:,:] which has added zero rows top and bottom, but dimension 2 still has 3 elements. And so forth …

So it’s just a question of interpreting correctly what you are seeing. To make it easier to see, we can do what I suggested above:

for ii in range(Apad.shape[2]):
    print(f"Apad[:,:,{ii}] = \n{Apad[:,:,ii]}")

That gives this:

Apad[:,:,0] = 
[[  0.   0.   0.   0.]
 [  0.   0.  10.   0.]
 [  0. 100. 110.   0.]
 [  0. 200. 210.   0.]
 [  0.   0.   0.   0.]]
Apad[:,:,1] = 
[[  0.   0.   0.   0.]
 [  0.   1.  11.   0.]
 [  0. 101. 111.   0.]
 [  0. 201. 211.   0.]
 [  0.   0.   0.   0.]]
Apad[:,:,2] = 
[[  0.   0.   0.   0.]
 [  0.   2.  12.   0.]
 [  0. 102. 112.   0.]
 [  0. 202. 212.   0.]
 [  0.   0.   0.   0.]]

That makes it clear that it’s actually correct if we look at the appropriate dimensions, but the difficulty is that is not the default behavior of the print function.