Week 1 Assignment 1 conv_forward (a small pitfall)

Hello,

I was looking at this one for a while before I finally had the clarity to believe I was wrong. Funny how that works out, haha!

Some people may be familiar with the following…

# for ITER in range(start, stop, step):

I was able to resolve my errors fairly quickly when I realized that “stepping” as I’ve implemented it above and when used at a certain point ( or points) in the starter code would imply striding over the entries of the output.

What the function requires is striding the input indices to map to consecutive output indices.

So, if it helps anyone out there, don’t think…

[Input Side] → [Output Side]
0 → 0
1 → 1 * step
2 → 2 * step
3 → 3 * step
(etc.)

think…

0 → 0
1 * step → 1
2 * step → 2
3 * step → 3
(etc.)

I’ve tried keeping the idea somewhat abstract to not just give the whole thing away.
If this winds up saving anyone 45+ minutes of debugging, then I’ll know I’ve done my part!

19 Likes

Thanks for your suggestion.

Thanks for describing your thought process here. It makes sense. When I use words to describe what happens in conv_forward, the way I say it is: remember that you are looping over the output space. Then within the loops, the work is to calculate where you are in the input space that corresponds to the current position in the output space. The loops must cover every position of the output space. The striding happens in the input space.

6 Likes

Thank you very much for pointing that out. Although I had a different error, your message actually got me to check the variables I used to calculate the indices… turns out, I was using i where i should have used h. That one kept me frustrated for several weeks. Thank you!

2 Likes

I was using h where I should be using w haha! Thanks guys I was convinced I was right but as usual it was a typo!

1 Like

Hi, I have a small question…my code passed all tests however I dont seem to understand where the stride is accounted for since we are iterating the slices’ vertices one by one without skipping 2 when the stride=2 for example…I hope my question is clear enough:)

1 Like

I think the key point is what I mentioned in this earlier reply on this thread. The loops in conv_forward and pool_forward are over the output space. Your point is exactly correct that we have to visit every point in the output space to provide a value. But the thing to realize is that the striding happens in the input space, right? There we don’t necessarily step one by one. That is the purpose of the stride.

3 Likes

Yes I understand, but I think my code does not take the stride into consideration when taking the input slice however it passed all tests which confuses me…in which step exactly should I have included it?

1 Like

Update: as of February 2022, the tests have been enhanced to catch stride bugs in conv_forward.

The tests here do not catch this bug. That is a bug in the tests, which we have reported to the course staff, but they have not fixed it yet. The tests do not throw an error, but if you carefully examine your output values for the stride = 2 test case, you will see that not all of them agree.

The loops are over samples, h, w and c, right? So it is a 4 level nested loop. In the logic for handling each h and w value, you need to use the stride value to compute where that output location in the image maps to in the input image. Here’s a post from Tmosh that gives the formulas.

2 Likes

Thanks a lot! Very helpful tips although I’ve already spent more than 45 mins on this…

As an additional note, keep an eye on the range of the loop and make sure it is compatible with the output matrix shape.

1 Like

THANK YOU FOR THIS! I had been stuck at this for an hour now, turns out I was making the same mistake. Your reply got me to check :+1:

1 Like

This is a crucial point that I’ve been missing for an hour or three. Up until reading this, I was trying to stride on the output instead of on the input - this executed without errors but was not giving me the right Z.mean().

1 Like

Glad to hear that you found the right post to get you past that issue. The search engine in Discourse actually works really well, unlike in the old Coursera Forums. The new Discourse forums have been in use for almost a year now, so there’s a lot of value in the saved history. You can pretty well assume that any problem you hit has been hit by others before you.

Nice work! Onward! :nerd_face:

1 Like

stride was accounted for in the formula for calculating the shape of the output. we are looping through the output shape, so stride was included.

1 Like

Hi guys,

Referring to the convolution calculation:

Shouldn’t there be an if condition to break out of the for loop when the outer range of the slicing operation is > the padded image dimension?

Regards
Nazih

1 Like

No, if you need that, it means your code is incorrect. Your shapes and the loop limits should match if you’ve written it correctly. There are a number of posts earlier on this thread which discuss the point. Here’s one that’s a good starting point. And here’s another thread that has a nice description of the way all the loops work in conv_forward.

One other thing to note is that it is a mistake to include the stride as a “step” in the loop limits. That is covered at least implicitly on both the threads that I linked above.

1 Like

Noted. Thanks @paulinpaloalto for your feedback.
I think I can make the range stop at the right point without an if statement.

I will go over what you shared.

Thanks
Nazih

1 Like

Thanks a lot mate. I’ve spent hours debugging whats wrong and I wish I had seen your post earlier. The mistake I made was, I was adding the stride instead of multiplying. So if anyone worked the wrong logic as me please multiply the stride with h or w

1 Like