DLS4 week 1 exercise 3 .. again

Course Q&A Deep Learning Specialization Convolutional Neural Networks
1

There error i’m working with is a very common one on this forum I am aware.

ValueError Traceback (most recent call last)
in
6 “stride”: 2}
7
----> 8 Z, cache_conv = conv_forward(A_prev, W, b, hparameters)
9 z_mean = np.mean(Z)
10 z_0_2_1 = Z[0, 2, 1]

in conv_forward(A_prev, W, b, hparameters)
79 weights = W[:,:,:,c]
80 biases = b[:,:,:,c]
—> 81 Z[i, h, w, c] = conv_single_step(a_slice_prev, weights, biases)
82
83 # YOUR CODE ENDS HERE

in conv_single_step(a_slice_prev, W, b)
24 # YOUR CODE STARTS HERE
25
—> 26 s = a_slice_prev * W
27 Z = np.sum(s)
28 Z = Z + np.float64(b)
ValueError: operands could not be broadcast together with shapes (3,3) (3,3,4).

I will try to mention a few of the things that have been in these other posts and explain my reasoning.

To begin, “a slice prev” is a since slice in the for loops of the ‘c’ index. I’ve tried calculating a slice prev by taking the padded version of a prev, a prev pad, and using the index of vertical start and end, horizontal start and end, and the c channel.

To move forward I must go backward a step. How did I get a prev pad, this came from iterating over the ‘i’ samples of A prev pad. A prev pad came from A prev using the padding function. A prev was a 4D array (1,2,3,4) so A prev pad was also 4D (1,2,3,4). a prev pad came from iterating the ith dimension of A prev pad. This was the first index, index zero. a prev pad would be the varying indexes of A prev pad(i,2,3,4) a single slice of the different m samples.

Here I find my first slight confusion. At first I think if a prev pad is the ith iteration of A prev pad I might say it was also a 4D array. But maybe it could also be a 3D array since it itself doesn’t have an ith dimension of m, it just is a single slice of that stack of many a prev pads. So I’m not sure about 3 vs 4 dimension when looking at a a prev pad. I think 3.

Now to go back to where we started. I have a singe slice of index i of m samples called a prev pad. I then take this and use the vert start/end, horiz start end, and c to get a slice prev. There are three dimensions there with (vertstart:end, horiz start end, and c) which leads me again to believe a prev pad is 3D, this would then make a slice prev 3D.

But just like earlier going from A prev pad to a prev pad I selected for one index. I am iterating over the c channel of a prev pad to get a slice prev so maybe a slice prev is now just 2D since each one is just a 2D height and width of channel c?

My weights and biases are set to 4D with c being the 4th input. Z(i,h,w,c) I used a previous function conv single step which takes a slice prev, weights, and biases. so weights is 4D, a slice prev is 2D? or maybe 3. so multiplying a slice prev * weights is already gonna be a mistake there. Z in the original function output a scalar of the array x W + b. here Z is some form of (i,h,w,c)? so is it 4 dimensional array or can it be a scalar and that just means its Z as a function of thsoe 4 variables?

my original error being cant broadcast together 3,3 and 3,3,4 and the first arrow above the message is to s = a slice prev * W which leads me to believe I’ve somehow covered the mistake here because I see that a slice prev is probably 2D maybe 3D as it is a slice an ith sample of a cth channel leaving height and width. and my weights are a 4D array of fxfxn_c_prev x n_c. and W is the channel index of that so it would be 3D maybe. so a slice_prev (2D) x W(3D) would seem to be the problem but as I’ve explained the way I came to those values seems to make sense to me a bit. I may be misunderstanding how these arrays change shape as I select for individual samples(m) and channels(c)

Sorry for the long post but I wanted to clarify that I’ve looked through the forum and worked through the problem so just saying “your Z is the wrong shape” although helpful, is not something I haven’t thought through so I think theres something fundamental I’m missing. I also haven’t figured out how to put in print statements that actually show up in the output. I try print statements and my errors just come out and I don’t see anything printed. so i’m doing all of this without printing out any of the various w.shape, z.shape type stuff.

Thanks

2

When I change my definition for a slice prev from a prev pad(vert start/end, horiz start/end, c) to a prev pad(vert start/end, horiz start/end, colon) I change my error message to

IndexError Traceback (most recent call last)
in
6 “stride”: 2}
7
----> 8 Z, cache_conv = conv_forward(A_prev, W, b, hparameters)
9 z_mean = np.mean(Z)
10 z_0_2_1 = Z[0, 2, 1]

in conv_forward(A_prev, W, b, hparameters)
79 weights = W[:,:,:,c]
80 biases = b[:,:,:,c]
—> 81 Z[i, h, w, c] = conv_single_step(a_slice_prev, weights, biases)
82
83 # YOUR CODE ENDS HERE

IndexError: index 4 is out of bounds for axis 3 with size 4

3

The only theory I can come up with for that is that your print statements are after the point at which the exception gets thrown. If you print things logically before the first call to conv_single_step, then the print statements should show up.

1 Like
4

Okay yeah i just figured out the print statements a few minutes ago. My room mate helped. I get:
biases:
(1, 1, 1)
weights:
(3, 3, 4)
a slice prev:
(3, 3, 4)

1 Like
5

That looks right. But I’m guessing that you now get the second error message about

80 biases = b[:,:,:,c]
—> 81 Z[i, h, w, c] = conv_single_step(a_slice_prev, weights, biases)
82
83 # YOUR CODE ENDS HERE

IndexError: index 4 is out of bounds for axis 3 with size 4

The point of that one is that the shape of your Z value does not match the indices of your loop. And it is the “channel” axis that is the problem, right? I added some prints to my code and here’s what I see from the test cell for conv_forward:

stride 2 pad 1
New dimensions = 3 by 4
Shape Z = (2, 3, 4, 8)
Shape A_prev_pad = (2, 7, 9, 4)
Z[0,0,0,0] = -2.651123629553914
Z[1,2,3,7] = 0.4427056509973153
Z's mean =
 0.5511276474566768
Z[0,2,1] =
 [-2.17796037  8.07171329 -0.5772704   3.36286738  4.48113645 -2.89198428
 10.99288867  3.03171932]
cache_conv[0][1][2][3] =
 [-1.1191154   1.9560789  -0.3264995  -1.34267579]
First Test: All tests passed!
stride 1 pad 3
New dimensions = 9 by 11
Shape Z = (2, 9, 11, 8)
Shape A_prev_pad = (2, 11, 13, 4)
Z[0,0,0,0] = 1.4306973717089302
Z[1,8,10,7] = -0.6695027738712113
stride 2 pad 0
New dimensions = 2 by 3
Shape Z = (2, 2, 3, 8)
Shape A_prev_pad = (2, 5, 7, 4)
Z[0,0,0,0] = 8.430161780192094
Z[1,1,2,7] = -0.2674960203423288
stride 1 pad 6
New dimensions = 13 by 15
Shape Z = (2, 13, 15, 8)
Shape A_prev_pad = (2, 17, 19, 4)
Z[0,0,0,0] = 0.5619706599772282
Z[1,12,14,7] = -1.622674822605305
Second Test: All tests passed!

What are your Z dimensions?

6

Z.shape is 2,5,7,4, so thats 4D and needs to be 3D?

7

No, it needs to be 4D. The dimensions are samples x height x width x channels. But the question is what those dimension values are for height and width and output channels. It looks like you just used the shape of the input A_prev, but that is wrong. You have to calculate the output shape using the formula:

n_{out} = \displaystyle \lfloor \frac {n_{in} + 2p - f}{s} \rfloor + 1

Also note that your output channels are wrong.

8

When I tried zeroing using the items from the explanation paragraph n_H, n_W, n_C, and even adding m in it says that you can’t zero with those float 64 items and np.int doesn’t work on them either. I’ve tried np.zeros(m,n_H, n_W, n_C) as well as m, np.zeros(nh, nw, nc) to fit the three argument limit and it brings up the float64 problem.

9

what do you mean by output channels? like ‘c’ channels I should use n_C instead of c?

10

The problem is that you need to understand the API of np.zeros. It takes a single argument which is a python “tuple” giving the requested shape, right? You can find the documentation by googling “numpy zeros”. We’ve used it lots of times in DLS up to this point …

11

Look at the shape of W. It is f x f x nC_{in} x nC_{out}, right? So that means there are nC_{out} filters, each of which match the number of channels in the input, which is nC_{in}. So the output will have nC_{out} channels.

12

Yes, I always have the np zeros since joining this class. so I just did a simple np.zero 3,3,4 to match my W shape and it says too many indices for array now. but my Z shape matches my W shape and a prev slice.

13

Yes, I believe n_C is my output of those channels. Should I use this for zero initializing Z?

14

Well the shape of Z only takes one of its dimensions directly from the shape of W, right? That’s the output channels. The rest of the dimensions of Z are:

The samples dimension, which is m from A_prev.

The height dimension, which you have to calculate using the formula I showed above.

The width dimension, which is like height: you need to calculate.

15

Right I see what youre saying. So I tried initializing Z a different way. I used zeros(nH, hW, nC) dtype=int) to try and bypass the float 64 problem. It still popped up. I understand that I’m missing ‘m’ still but i’m trying to solve one problem at a time. So even with dtype = int I get the float 64 problem.

16

TypeError Traceback (most recent call last)
in
6 “stride”: 2}
7
----> 8 Z, cache_conv = conv_forward(A_prev, W, b, hparameters)
9 z_mean = np.mean(Z)
10 z_0_2_1 = Z[0, 2, 1]

in conv_forward(A_prev, W, b, hparameters)
38 # Initialize the output volume Z with zeros. (≈1 line)
39 # Z = None
—> 40 Z = np.zeros((m,n_H, n_W, n_C), dtype = int)
41
42 # Create A_prev_pad by padding A_prev

TypeError: ‘numpy.float64’ object cannot be interpreted as an integer

17

Are you suggesting I not use terms like m and n_C directly but rather index into W and A_prev for them?

18

This is what happens when I define Z by indexing other variables.

IndexError Traceback (most recent call last)
in
6 “stride”: 2}
7
----> 8 Z, cache_conv = conv_forward(A_prev, W, b, hparameters)
9 z_mean = np.mean(Z)
10 z_0_2_1 = Z[0, 2, 1]

in conv_forward(A_prev, W, b, hparameters)
38 # Initialize the output volume Z with zeros. (≈1 line)
39 # Z = None
—> 40 Z = np.zeros((A_prev[0],n_H, n_W, W[3]), dtype = int)
41
42 # Create A_prev_pad by padding A_prev

IndexError: index 3 is out of bounds for axis 0 with size 3

1 Like
19

That error must mean that one of the “tuple” values you are passing to np.zeros is a float instead of an integer. So how could that happen? BTW you don’t need to set the output type of np.zeros. Try

print(type(n_H))

and so forth. You are giving the numbers for dimensions, so they need to be integers.

20

Try this:

print(A_prev[0])

and see what you get. Hint: it will be a 3 dimensional numpy array of floats. I don’t think that’s what you want.

It’s late my time, so I’m going to sign off for tonight.