In Conv2D layer, the padding parameter has only two options: same or valid.
same: same input and output size
valid: no padding at atll
But aren’t these two special cases of padding? For example, for an input n x n x 3 image, if padding=1 or 2 or 3 are used, these paddings are not necessarily SAME paddings. When p != (f-1)/2, this won’t be same padding. Is this (not same padding) not used in practice?
I think the point of padding is to keep the shape unchanged after convolution, so same enables it and valid does not enable it, and that’s it!
Your question makes sense because we can add more paddings than it is necessary to keep the shape, however, would we do it on purpose? Remember that the padding alone and by itself does not give us any information because the padded values are just zeros, and what do zeros help train a model that makes better prediction? Is there even any justification behind? I think a justification is what we need when we consider to go beyond what people usually do.
If your justification is that I must make additional paddings for your own purpose, then why not? However, to do so, perhaps you just need an extra layer for it.
Cheers,
Raymond
Regarding this:
tf.keras.layers.ZeroPadding2D( padding=(1, 1), data_format=None, **kwargs)
The default padding=(1,1). To make a SAME padding, p must be (f-1)/2, which means if I need a SAME padding, I need to calculate a proper “padding” parameter in considering the ‘f’ value. Is that right? If padding is either valid or same, Is it more appropriate to have a default padding=None than (1,1)?
Here is how I will use it, if I just want to achieve ‘same’, then I would set it in tf.keras.layers.Conv2D. If I, for some reason, want to add extra padding that is not offerable by Conv2D, then I would use tf.keras.layers.ZeroPadding2D.
Yes
Is the behavior for None properly defined in tf.keras.layers.ZeroPadding2D? What is that behavior?
If I don’t need same padding, the effect of the padding can only be seen through evaluation, i.e. accuracy? Otherwise, it would be hard to see whether it has any value. In the conv_forward implementation, A_prev_pad is first applying zero_padding. In that case, is that for same padding or just some other padding for other purpose?
For padding=None, I thought it says no padding will be used.
I think we add padding because we know we need it to keep the shape after convolution. I would not add it and then ask what effect a padding can give us. Padding is not introduced as a tool to boost model performance at all.
If I don’t want padding, I will not use tf.keras.layers.ZeroPadding2D in the first place.
So if I need to keep the same shape, then I need to use padding, either through SAME parameter, or a proper padding value calculated based on f. Otherwise, no padding will be used. So the use of padding needs to be considered along with the whole network design. Otherwise, it is hard to know whether a padding is necessary.
Exactly! Very well summarized!
Raymond