# Batch Normalization on Channels

**URL:** <https://community.deeplearning.ai/t/batch-normalization-on-channels/132251>\
**Category:** Convolutional Neural Networks\
**Tags:** coursera-platform\
**Created:** [May 28, 2022, 11:32am UTC](https://community.deeplearning.ai/t/batch-normalization-on-channels/132251 "2022-05-28T11:32:35Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ayse\_Burcu\_Ozkaptan](https://avatars.discourse-cdn.com/v4/letter/a/ee7513/32.png) [@Ayse\_Burcu\_Ozkaptan](https://community.deeplearning.ai/u/Ayse_Burcu_Ozkaptan)\
**Post date:** [May 28, 2022, 11:32am UTC](https://community.deeplearning.ai/t/batch-normalization-on-channels/132251/1 "2022-05-28T11:32:35Z")

</div>

Hello everyone,  
Why do we have to run the normalization on channels axis? If the input, say an RGB image, has 3 channels, wouldn’t it be a better approach to normalize the values on each channel separately (i.e. normalization across axes 1 and 2)? or am I getting it wrong?  
Cheers,

---

<div class="post-metadata">

**Author:** ![balaji.ambresh](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/balaji.ambresh/32/128697_2.png) [@balaji.ambresh](https://community.deeplearning.ai/u/balaji.ambresh)\
**Post date:** [May 28, 2022, 11:59am UTC](https://community.deeplearning.ai/t/batch-normalization-on-channels/132251/2 "2022-05-28T11:59:37Z")

</div>

For each image that passes through a `Conv2D` layer, the following is the output dimension:`[new_height, new_width, num_filters_in_conv]`. The last dimension corresponds to channels. When you run batch norm across the channel axis, for every channel, batch norm tracks 4 variables: \beta, \gamma, \mu, \sigma. Please see [this](https://www.tensorflow.org/api_docs/python/tf/keras/layers/BatchNormalization) link.

---

<div class="post-metadata">

**Author:** ![anon57530071](https://avatars.discourse-cdn.com/v4/letter/a/46a35a/32.png) [@anon57530071](https://community.deeplearning.ai/u/anon57530071)\
**Post date:** [May 28, 2022, 12:39pm UTC](https://community.deeplearning.ai/t/batch-normalization-on-channels/132251/3 "2022-05-28T12:39:26Z")

</div>

I suppose Balaji explained key points already, but here are few additions.

BatchNormalization is to apply “normalization and scale/shift” to the output of previous layer in order to stabilize the neural network. On the other hand, filters which are used in Conv2D (and others) are trying to extract characteristics of image/text. Each filter creates one output, a channel. In this sense, as you know, the number of channels in the output is equal to the number of filters. There will be no RGB channel even from the first Conv2D function.  
Then, the next question may be,… is there any advantages or meanings to handle a single RGB signal (say, R) separately ? I think, to extract characteristics from an image, combinations of RGB have more importance than a single channel data. Applying a single filter to all three channels and get one output (channel) should have more meaningful characteristics which can be utilized for image detections and others.

---

<div class="post-metadata">

**Author:** ![Ayse\_Burcu\_Ozkaptan](https://avatars.discourse-cdn.com/v4/letter/a/ee7513/32.png) [@Ayse\_Burcu\_Ozkaptan](https://community.deeplearning.ai/u/Ayse_Burcu_Ozkaptan)\
**Post date:** [May 28, 2022, 1:01pm UTC](https://community.deeplearning.ai/t/batch-normalization-on-channels/132251/4 "2022-05-28T13:01:01Z")

</div>

Thank you very much for your response! However, this was not actually what I was looking for. I was wondering why we are normalizing across channels rather than the other dimensions of the convolution output, namely normalizing across the height and width?

---

<div class="post-metadata">

**Author:** ![anon57530071](https://avatars.discourse-cdn.com/v4/letter/a/46a35a/32.png) [@anon57530071](https://community.deeplearning.ai/u/anon57530071)\
**Post date:** [May 28, 2022, 1:53pm UTC](https://community.deeplearning.ai/t/batch-normalization-on-channels/132251/5 "2022-05-28T13:53:13Z")

</div>

This is actually an interesting question. I suppose there are some other research works to find the group to normalize. Batch norm, Layer norm, Instance Norm, Group Norm,…  
Here is one figure from the paper: [Yuxin Wu, Kaiming He. “Group Normalization”](https://arxiv.org/pdf/1803.08494.pdf)

 ![image](https://global.discourse-cdn.com/dlai/original/3X/a/b/abd2ea2cc52f6ab78af7209e52b6b7b8c2f9896c.png)

In some cases, authors could get a better result with Group Normalization. If you are interested in, please take a look at this interesting paper. Hope this helps.
