Andrew mentions on the second slide of the “backpropagation intuition video” that : " dz^1 is equal to w^2 transpose times dz^2 and then, times an element-wise product of g^1 prime of z^1."
but I don’t understand where this formula is coming from.
is this demonstrated on another slide and I missed it?
many thanks for your help
Please follow the chain rule in calculus. The programming assignment for the week also covers the equations of backpropagation.
many thanks for your reply.
i did the programing assignment, and a little work with the chain rule (i’m somewhat familiar with algebra) but unfortunately i still cannot find why " dz^1 is equal to w^2 transpose times dz^2 and then, times an element-wise product of g^1 prime of z^1"…
in week 4 , “forward and backward propagation” Andrew gives afew hints (see the green line in the slide), but then we don’t know where the equation: dZ[l]=da[l]*g[l]'(z[l]) is coming from. is there, somewhere, a step-by-step demonstration of this very important formula?
any idea?
thanks much!
We want to compute
dz^{[1]} = \frac{\partial{L}}{{\partial{z^{[1]}}}}
Using chain rule:
\frac{\partial{L}}{{\partial{z^{[1]}}}} = \frac{\partial{z^{[2]}}}{{\partial{a^{[1]}}}} \frac{\partial{L}}{{\partial{z^{[2]}}}} \frac{\partial{a^{[1]}}}{{\partial{z^{[1]}}}}
Since z^{[2]} = W^{[2] T} a^{[1]} + b^{[2]}
we get \frac{\partial{z^{[2]}}}{{\partial{a^{[1]}}}} = W^{[2] T}
We know that \frac{\partial{L}}{{\partial{z^{[2]}}}} = dz^{[2]}
We represent
\frac{\partial{a^{[1]}}}{{\partial{z^{[1]}}}} = g^{[1]prime} (z^{[1]}) (sorry. \prime symbol isn’t rendering properly)
Substitute the elements back in chain rule representation and you have it.
many thanks for your answer!!