Keras - number of parameters in BatchNorm Layer

How to calculate the number of parameters in a BatchNorm-Layer?

Hi Andreas,

The BatchNorm-Layer only performs a normalization operation. The output shape is the same as the input shape. See tf.keras.layers.BatchNormalization  |  TensorFlow Core v2.4.1

Hi Mentor, thank you very much for answering

I know it’ s doing normalization (and there 's a gamma and a beta…)
But here the pasted model.summary says there are 128 parameters in the batch norm-layer …
I am still wondering…

Greetings
Andreas

zero_padding2d_4 (ZeroPaddin (None, 70, 70, 3) 0


conv0 (Conv2D) (None, 64, 64, 32) 4736


bn0 (BatchNormalization) (None, 64, 64, 32) 128


activation_4 (Activation) (None, 64, 64, 32) 0


Hi Andreas,

They are the gamma and beta weights, as well as the mean and std (see, e.g., keras - How the number of parameters associated with BatchNormalization layer is 2048? - Stack Overflow and keras - How to set weights of the batch normalization layer? - Stack Overflow).

The number of each parameter is equal to the number of features, which in this case is 32. So you get 4*32 = 128 parameters.

If you want to dive deeper into this you can have a look at this paper:
jmlr.org/proceedings/papers/v37/ioffe15.pdf

2 Likes

I guess I got it

Thank you very much