In this assignment we’re asked to implement ResNet. One of the steps is to add a batch normalization using Keras. The command given includes an axis command that indicates the feature to be normalized: BatchNormalization(axis = 3)(X, training = training). May I ask what feature are we normalizing here when axis is set to 3 and why are we normalizing this axis/feature? Thanks.
Hi @zhexingli , because the code is using Tensorflow as backend, we have four parameters and so, to make sure that we are normalizing the number of channels, we set the axis=3.
1 Like
Hello @zhexingli!
In addition to Carlos reply, we do normalization to make the training more stable and fast, so model performed better. If you know what is normalization, then it will be easy for you to understand why we do that and how it makes the calculation stable and faster and avoid vanishing and exploding gradient problems.
Best,
Saif.
1 Like