Week 1 -Possible Mistake on Lecture Video?

Maybe this is just a language issue, but I would not describe the way dropout works as “downsizing” anything. What it does it randomly disable the output of some neurons by setting their output values to zero in the layers to which you apply dropout. It is applied at the output of the layer, by multiplying the activation output of the layer, A^{[l]} by a “mask” matrix in which the elements are 1 or 0. The “keep probability” determines the probability that a given element of the mask matrix is 1.

You can apply dropout at multiple layers in the way that I just described.

The exact elements that are zeroed out by the mask in a given layer changes randomly with each training iteration. Note that another subtlety is that the matrix A^{[l]} contains multiple columns, each of which is the output for one “sample”. The elements of each column that get “zapped” are also different for each sample in a given iteration.

The effect that dropout has is to weaken the dependence of the later layers of the network on specific neurons in the given layer where dropout is applied. Note that it works both during forward and backward propagation.

This was all described in detail in the lectures. If what I said above does not make sense, it might be a good idea to just go back and listen the dropout lectures again.

There are also a number of existing threads that discuss all this in more detail. Here’s one that delves into the point I mentioned above about the “zapping” being different per sample.

Here’s a thread with a number of posts that discusses the rationale for the factor of 1/keep_prob that you see in the dropout computation.