Normalization in keras

norm_l = tf.keras.layers.Normalization(axis=-1)

In the above line what does axis=-1 means ?

Hi @Ashutoshsahu ,

Welcome to the community! This is your first post :slight_smile:

axis = -1 means that the index that will be returned by Normalization will be taken from the last axis.

For instance, in a shape like (5, 10, 15): axis=-1 would normalize (5, 10,…) from 0 to 14.

Does it make sense?

Juan

I did not get it.
Correct me if I am wrong
1.) shape (5,10,15) means we have 5 matrices of 10 rows and 15 columns ?

a.) What does normalize( 5,10,…) from 0 to 14 means. How you determined the numbers 0 and 14. Why not it can be normalize between 6 to 18?

I understand normalization. To normalize x we do (x−μ)/σ.
But I am not able to understand how this is applied here (in the above code that I asked)?

Hello @Ashutoshsahu,

Let’s take a step back and look at this example:

ary = np.array([
    [ [1, 2, 3,], [2, 3, 4,], ],
    [ [5, 4, 3,], [3, 2, 1,], ],
])

This array has a shape of (2, 2, 3) because ary has only 2 elements which are two 2 x 3 arrays. On the other hand, this is a 3D array because it has 3 axis. The 0th axis has a size of 2; 1st axis a size of 2; and the 2nd (or the -1th) axis has a size of 3. Again, in summary, it is (2, 2, 3)

I would say it has a shape of (2, 2, 3) instead of 2 matrices of 2 rows and 3 columns. If I have to describe it in English, I would instead say 2 tables of something or 2 pages of something. However, my preference is “a shape of (2, 2, 3)”.

Applying the normalization layer on my example array along axis -1 means to apply it along the axis 2. Because the 2nd axis is the last axis. What @Juan_Olano meant, but in my example, is that it would calculate normalization constants for (2, 2, 0) first, and then (2, 2, 1). If it is an array of shape (5, 10, 15), then it would first calculate for (5, 10 , 0), then (5, 10, 1), then (5, 10, 2), … until (5, 10, 14).

You may also want to read the doc about the meaning of axis.

Cheers,
Raymond