# Inverted Dropout **URL:** \ **Category:** Improving Deep Neural Networks: Hyperparameter tun\ **Tags:** coursera-platform\ **Created:** [June 9, 2021, 7:04am UTC](https://community.deeplearning.ai/t/inverted-dropout/12940 "2021-06-09T07:04:14Z")\ **Posts on this page:** 3\ **Page:** 2
**Author:** ![Sahil\_RB](https://avatars.discourse-cdn.com/v4/letter/s/4bbf92/32.png) [@Sahil\_RB](https://community.deeplearning.ai/u/Sahil_RB)\ **Post date:** [August 2, 2022, 11:27am UTC](https://community.deeplearning.ai/t/inverted-dropout/12940/21 "2022-08-02T11:27:16Z")
> [@paulinpaloalto](#): > > Even though we are zapping some neurons in the layer each iteration, we want the “expected value” of the activations viewed at the aggregate level to stay roughly constant. I understand that dividing by keep\_prob will help in scaling up the value of the activations by roughly the same amount that they have been reduced due to the removal of some nodes. But why do we need this value to stay roughly constant in the first place? ---
**Author:** ![paulinpaloalto](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/paulinpaloalto/32/187_2.png) [@paulinpaloalto](https://community.deeplearning.ai/u/paulinpaloalto)\ **Post date:** [August 2, 2022, 3:08pm UTC](https://community.deeplearning.ai/t/inverted-dropout/12940/22 "2022-08-02T15:08:42Z")
The reason is that all forms of regularization, including dropout, only happen during _training_. When we actually use the trained network to make predictions, there will be no dropout (which may be achieved without changing the code by simply setting _keep\_prob = 1._) So if you don’t scale up to compensate for the dropout, then the later layers will be trained in a way that won’t agree with what they actually are getting when we run normal prediction. They will be trained to expect lower aggregate “energy” from the previous layers, so things may not work as well as expected in normal “prediction” mode without dropout. It’s been a while since I have watched Prof Ng’s lectures on this topic, so I don’t remember exactly what he says about this, but he must discuss this point. It might be worth watching again to see. (_Update_: yes, he mentions this between 8:30 and 9:00 in the [main lecture on Dropout.](https://www.coursera.org/learn/deep-neural-network/lecture/eM33A/dropout-regularization)) ---
**Author:** ![paulinpaloalto](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/paulinpaloalto/32/187_2.png) [@paulinpaloalto](https://community.deeplearning.ai/u/paulinpaloalto)\ **Post date:** [July 27, 2023, 6:17pm UTC](https://community.deeplearning.ai/t/inverted-dropout/12940/23 "2023-07-27T18:17:00Z")
[Previous page](https://community.deeplearning.ai/t/inverted-dropout/12940.md?page=1)