Question about Dropout in the Discriminator: Lab2 & Assignment

In both, Lab2 and the programming assignment we use Dropout Layers. My question is if it would not be more convenient to use a tf.keras.layers.SpatialDropout2D as it is mentioned:

“This version performs the same function as Dropout, however, it drops entire 2D feature maps instead of individual elements. If adjacent pixels within feature maps are strongly correlated (as is normally the case in early convolution layers) then regular dropout will not regularize the activations and will otherwise just result in an effective learning rate decrease. In this case, SpatialDropout2D will help promote independence between feature maps and should be used instead.”

Hello again Peter,

The main reason I would think why they didn’t use SpatialDropout 2D is that it drops entire 2D feature maps instead of individual elements even though performs the same function as Dropout.

I hope you have noticed in both the lab2 this part in the generator description:
Reshape the output to have three dimensions. This stands for the (length, width, number of filters).

This is from Assignment ===input_shape takes in a list containing the random normal dimensions.

random_normal_dimensions is a hyperparameter that defines how many random numbers in a vector you’ll want to feed into the generator as a starting point for generating images. So basically GAN model is not looking for a dropout layer which would drop the entire 2D feature maps rather individual elements randomly.

Feature matching changes the cost function for the generator to minimize the statistical difference between the features of the real images and the generated images. So use of Spatialdropout2D will cause selective dropout of entire 2D feautres which is not the main goal here.

feature matching expands the goal from beating the opponent to matching features in real images. The means of the real image features are computed per minibatch which fluctuate on every batch. It introduces randomness that makes the discriminator harder to overfit itself.