Hello,
I have been trying to calculate dz[1] manually and I got this expression: dz[2] x dw[2] x g[1]'(z[1]). However, I do not really understand where the element-wise operation (*) comes from. If anyone could help me, I would appreciate it.
Thanks in advance.
Hello Julio,
is this related to any assignment in particular?
Can you please mention the week and assignment name you are working on?
Regards
DP
Hello,
This issue appears in week 3, in the explanation of backward propagation.
Do you have an understanding about what element-wise operation is?
Yes, I understand that the element-wise product consists in multipliying each element without adding later. But I do not really understand why the operation is element-wise and not a dot product.
Hi Julio,
Were you able to get some clarity on this point? I am currently in Wk-3 of the course and have the same confusion around why the element-wise product has been used for dZ[1].
Thank you,
Sunetra
It’s Calculus. The activation function g^{[l]} is applied element-wise to
z^{[l]}, so its derivative is also applied element-wise.
Dot product and elementwise multiply are completely different operations, of course. It depends on how the math works out. Some operations are “elementwise” in forward propagation, e.g. the application of the activation functions. So when you apply the Chain Rule to that operation, you will end up with elementwise multiplies.
Note that Prof Ng has specifically designed these courses not to require knowledge of calculus. That means he cannot show us the actual derivations of back propagation: they are literally beyond the scope of these courses. He just gives us the formulas and we take his word for it. If you have the math background to understand vector and matrix calculus, here’s a thread with links to resources that cover the derivation of back propagation.
Thank you Saif. Your one explanation is great and helped me understand the concept. 
Thank you, it is clear now 