# Batch Norm Backprop

**URL:** <https://community.deeplearning.ai/t/batch-norm-backprop/16348>\
**Category:** Improving Deep Neural Networks: Hyperparameter tun\
**Tags:** coursera-platform\
**Created:** [June 22, 2021, 9:22am UTC](https://community.deeplearning.ai/t/batch-norm-backprop/16348 "2021-06-22T09:22:35Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![MalayAgr](https://avatars.discourse-cdn.com/v4/letter/m/51bf81/32.png) [@MalayAgr](https://community.deeplearning.ai/u/MalayAgr)\
**Post date:** [June 22, 2021, 9:22am UTC](https://community.deeplearning.ai/t/batch-norm-backprop/16348/1 "2021-06-22T09:22:35Z")

</div>

I’ve been trying to derive the gradients for batch normalisation since I want to implement it on my own. I’ve successfully found the gradients with respect to beta and gamma, but can’t figure out the gradients for weights, biases (not removing them, just yet) and the dA for the previous layer (the layer preceding this, which which will use this for its own backprop step). The main problem I’m facing is figuring out the derivative of the mean and the standard deviation with respect to Z (linear combination with weight and bias).

I was wondering if anyone can help me?

---

<div class="post-metadata">

**Author:** ![nramon](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/nramon/32/129097_2.png) [@nramon](https://community.deeplearning.ai/u/nramon)\
**Post date:** [June 22, 2021, 11:56am UTC](https://community.deeplearning.ai/t/batch-norm-backprop/16348/2 "2021-06-22T11:56:16Z")

</div>

Hi, @MalayAgr.

[Here](https://chrisyeh96.github.io/2017/08/28/deriving-batchnorm-backprop.html)’s the derivation of the backprop equations for batch normalization.

There’s a [link](https://kevinzakka.github.io/2016/09/14/batch_normalization/) at the bottom to a blog post that includes a Python implementation.

Happy hacking 🙂

---

<div class="post-metadata">

**Author:** ![MalayAgr](https://avatars.discourse-cdn.com/v4/letter/m/51bf81/32.png) [@MalayAgr](https://community.deeplearning.ai/u/MalayAgr)\
**Post date:** [June 22, 2021, 3:45pm UTC](https://community.deeplearning.ai/t/batch-norm-backprop/16348/3 "2021-06-22T15:45:48Z")

</div>

Hi.

Thanks! I was able to figure it out using this very blog post. Interestingly, only the computation for dZ changes in the entire process. dW and db are still calculated in the same way. Pretty amazing how these things work out in the end.

---

<div class="post-metadata">

**Author:** ![Neurojedi](https://avatars.discourse-cdn.com/v4/letter/n/c68b51/32.png) [@Neurojedi](https://community.deeplearning.ai/u/Neurojedi)\
**Post date:** [May 4, 2022, 9:49pm UTC](https://community.deeplearning.ai/t/batch-norm-backprop/16348/4 "2022-05-04T21:49:13Z")

</div>

Hi,

I’m trying to do something similar and add batch normalization to L-layer neural network that we created in the first course. I would appreciate it if you could possibly share how you changed the dZ .