Dear Mentor,
Could you please guide me how to prove that this 2 backprop formula for CONV layer are valid?
Thank you.
That requires quite a bit of calculus. Not my strong suit, sorry.
I had the same question and found this youtube video pretty insightful. It explains how we get dW: https://www.youtube.com/watch?v=z9hJzduHToc. He assumes pad = 0, stride = 2. What he arrives at in the end generalizes to the formula listed here for dWc.
The video doesn’t cover how to calculate dA, but I was able to figure it out using the same logic. Taking the video’s example, dL/da1 = (dL/dz1).(dz1/da1). Here, we know dL/z1 from our calculation of dW above, and dz1/da1 will be w1 since z1 = w1a1 + w2a2 + …, etc. You should be able to figure out the rest and arrive at the formula for dA listed in the notebook.
Hope this helps.
Dear Mr Ashwin Ramakrishnan,
Thank you so much for your guidance.
1 Like