Does Dropout implementation shut down random features and not random neurons?

[nramon]

Hi, @navid.ai.

@navid.ai

Therefore, the neurons we shut down are different from one example to another.

Did I understand this correctly? If so, I was hoping to understand better why we don’t shut down a neuron for all examples?

Yes, this is correct. For each training example, you sample a thinned network by randomly dropping out neurons.

You’re trying to approximate the effect of averaging the predictions of many different neural networks to prevent overfitting (source), so you wouldn’t want to use the same network for all the training examples