Hi there,
After I go through the lecture and related problem sets, I still feel a bit confused about the 4th step of inverted dropout. What’s the meaning of this step? Can anyone elaborate a little bit on this step?
Thanks!
Hello @Zihan_ZHU,
Because we use dropout at training time but we do not use dropout at testing time. Without the correction of “divided by keep_prob”, the layer’s output at training will always be (e.g.) lower than the same layer’s output at testing. The correction attempts to compensate for that difference. It basically enlarge the contribution from each keep-neurons in compensation of the loss due to neurons dropping.
Raymond
1 Like
Hi @Zihan_ZHU ,
In addition to @rmwkwok clear explanation, I wanted to share a link to another post where this is also discussed: Post about inverted dropout.
I hope it adds value 
Juan
1 Like