# Please explain $dz^{\[1\]} = {W^{\[2\]}}^{T} dz^{\[2\]} \\times {g^{\[1\]}}^{'}(z^{\[1\]})$ in backpropogation **URL:** \ **Category:** Neural Networks and Deep Learning\ **Tags:** week-module-3, coursera-platform\ **Created:** [November 3, 2024, 4:09am UTC](https://community.deeplearning.ai/t/please-explain-dz-1-w-2-t-dz-2-times-g-1-z-1-in-backpropogation/717693 "2024-11-03T04:09:31Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![munish259272](https://avatars.discourse-cdn.com/v4/letter/m/c4cdca/32.png) [@munish259272](https://community.deeplearning.ai/u/munish259272)\ **Post date:** [November 3, 2024, 4:09am UTC](https://community.deeplearning.ai/t/please-explain-dz-1-w-2-t-dz-2-times-g-1-z-1-in-backpropogation/717693/1 "2024-11-03T04:09:31Z")
Why it is dz^{[1]} = {W^{[2]}}^{T} dz^{[2]} \times {g^{[1]}}^{'}(z^{[1]}) and not A^{[1]} - Y like it was done for dz^{[2]} = A^{[2]} - Y. ![image](https://global.discourse-cdn.com/dlai/original/3X/6/6/6617b9b9d2c5af50887dfd0ee4df0eccc5b95bb2.png) My understanding : - We are not doing A^{[1]} - Y because it is not directly connected to the true labels directly, instead error is being propogated/passed on backwards from layer 2(output layer) to layer 1 - dz^{[1]} = {W^{[2]}}^{T} dz^{[2]} \times {g^{[1]}}^{'}(z^{[1]}) means \frac{dLoss}{{dz^{[1]}}} = {W^{[2]}}^{T} \frac{dLoss}{{dz^{[2]}}} \times \frac{dA^{[1]}}{{z^{[1]}}}. - \frac{dA^{[1]}}{{z^{[1]}}} Represents how much A^{[1]} changes w.r.t z^{[1]} which(A^{[1]}) in turn is input to the output layer responsible for the loss \frac{dLoss} {{dz^{[2]}} }, so \frac{dA^{[1]}}{{z^{[1]}}} kind of scales the error(by multiplying \frac{dLoss}{{dz^{[2]}}} \times \frac{dA^{[1]}}{{z^{[1]}}}) in the output layer and the whole thing is multiplied by {W^{[2]}}^{T} to pass a portion of the error(contributed by layer2) to layer1. Please clarify and correct me. ---
**Author:** ![paulinpaloalto](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/paulinpaloalto/32/187_2.png) [@paulinpaloalto](https://community.deeplearning.ai/u/paulinpaloalto)\ **Post date:** [November 3, 2024, 4:56am UTC](https://community.deeplearning.ai/t/please-explain-dz-1-w-2-t-dz-2-times-g-1-z-1-in-backpropogation/717693/2 "2024-11-03T04:56:10Z")
Things work out differently at the output layer because of the way that the derivative of sigmoid and the derivative of the cross entropy loss function work nicely together. [Here’s a thread](https://community.deeplearning.ai/t/derivation-of-dl-dz/165) which shows that. At the inner layers of the network, you don’t get that nice simplification. But note that derivations involving calculus are beyond the scope of these courses. If you have some math background, [here’s a thread](https://community.deeplearning.ai/t/course-1-week-3-backpropagation-intuition/7979/2) with links to more information on the derivations of back propagation. ---
**Author:** ![munish259272](https://avatars.discourse-cdn.com/v4/letter/m/c4cdca/32.png) [@munish259272](https://community.deeplearning.ai/u/munish259272)\ **Post date:** [November 3, 2024, 10:57am UTC](https://community.deeplearning.ai/t/please-explain-dz-1-w-2-t-dz-2-times-g-1-z-1-in-backpropogation/717693/3 "2024-11-03T10:57:15Z")
Ok now i understand the derivation behind \frac{dLoss}{{dz^{[1]}}} = {W^{[2]}}^{T} \frac{dLoss}{{dz^{[2]}}} \circ \frac{dA^{[1]}}{{z^{[1]}}} \frac{d \text{Loss}}{d z^{[1]}} Can be expressed using the chain rule as: \frac{d \text{Loss}}{d z^{[1]}} = \frac{d \text{Loss}}{d z^{[2]}} \cdot \frac{d z^{[2]}}{d A^{[1]}} \cdot \frac{d A^{[1]}}{d z^{[1]}} 1. **\frac{d \text{Loss}}{d z^{[2]}}:** 2. **\frac{d z^{[2]}}{d A^{[1]}}:** - Since z^{[2]} = W^{[2]} A^{[1]} + b^{[2]}, the derivative of z^{[2]} w.r.t A^{[1]} is just W^{[2]} 1. **\frac{d A^{[1]}}{d z^{[1]}}:** \frac{d A^{[1]}}{d z^{[1]}} = \frac{d g^{[1]}(z^{[1]})}{d z^{[1]}} = g^{[1]'}(z^{[1]}) Putting these terms together, we get: \frac{d \text{Loss}}{d z^{[1]}} = \frac{d \text{Loss}}{d z^{[2]}} \cdot W^{[2]} \circ g^{[1]'}(z^{[1]}) Each term has the following shape: 1. **\frac{d \text{Loss}}{d z^{[2]}}:** This is the gradient of the loss with respect to the pre-activation z^{[2]} at the output layer. If there are n^{[2]} nodes in layer 2 and m training examples, then: 2. **W^{[2]}:** The weight matrix for the connection from layer 1 to layer 2. If layer 1 has n^{[1]} nodes and layer 2 has n^{[2]} nodes, then: 3. **\frac{d \text{Loss}}{d z^{[2]}} \cdot W^{[2]}:** 4. **g^{[1]'}(z^{[1]}):** This is the derivative of the activation function in layer 1 with respect to z^{[1]}. ---
**Author:** ![paulinpaloalto](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/paulinpaloalto/32/187_2.png) [@paulinpaloalto](https://community.deeplearning.ai/u/paulinpaloalto)\ **Post date:** [November 3, 2024, 3:20pm UTC](https://community.deeplearning.ai/t/please-explain-dz-1-w-2-t-dz-2-times-g-1-z-1-in-backpropogation/717693/4 "2024-11-03T15:20:29Z")
That’s a nice explanation. Yes, all this is just the Chain Rule in action with the added twist that the objects are vectors and matrices. Forward propagation is a huge function composition: a series of functions each one feeding its output as the input to the next layer function and the loss (vector valued) and cost (scalar average of the loss) are the last two layers. Then when you want to compute the derivative of the cost, you apply the Chain Rule, peeling the onion one layer at a time from the outside in, as you showed.