Notice that your Z[0,2,1] values are correct up to the point where they all start being zero. So that probably means that you are not managing the “channel” index values correctly. Here are my outputs on that first test with some added print statements:
stride 2 pad 1
New dimensions = 3 by 4
Shape Z = (2, 3, 4, 8)
Shape A_prev_pad = (2, 7, 9, 4)
Z[0,0,0,0] = -2.651123629553914
Z[1,2,3,7] = 0.4427056509973153
Z's mean =
0.5511276474566768
Z[0,2,1] =
[-2.17796037 8.07171329 -0.5772704 3.36286738 4.48113645 -2.89198428
10.99288867 3.03171932]
cache_conv[0][1][2][3] =
[-1.1191154 1.9560789 -0.3264995 -1.34267579]
First Test: All tests passed!
Notice that 4 of your output values are correct and then 4 are zero. Also notice that the input has 4 channels, but the output has 8 channels. So you may have used the input channel dimension as the output channel dimension.
Here’s a post that explains in words how all the loops work in conv_forward. If my theory above doesn’t play out, then please have a look at that thread and see if it sheds any additional light.