# Implementation keep\_prob in dropout **URL:** \ **Category:** Improving Deep Neural Networks: Hyperparameter tun\ **Tags:** coursera-platform\ **Created:** [July 3, 2021, 9:39am UTC](https://community.deeplearning.ai/t/implementation-keep-prob-in-dropout/18832 "2021-07-03T09:39:17Z")\ **Posts on this page:** 1\ **Showing post:** 2
**Author:** ![kampamocha](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/kampamocha/32/234_2.png) [@kampamocha](https://community.deeplearning.ai/u/kampamocha)\ **Post date:** [July 3, 2021, 2:35pm UTC](https://community.deeplearning.ai/t/implementation-keep-prob-in-dropout/18832/2 "2021-07-03T14:35:57Z")
Hi @jijo, Given the random nature of the process, that could happen. You have no guarantee that the droped neurons would be exactly 20%, it could be more or less depending on randomness. There have been some discussions about it in this forum, so I encourage you to take a look at those. Some posts that could be helpful are below, but I’m sure there’s more. > [@Week 1 - Doubt in Dropout Regularization lecture video](https://community.deeplearning.ai/t/week-1-doubt-in-dropout-regularization-lecture-video/14064/4): > > No need to apologize! The problem of symmetry is something [different](https://www.coursera.org/learn/neural-networks-deep-learning/lecture/XtFPI/random-initialization). Dividing by keep\_prob is compensating for this: [image] ([source](https://www.cs.toronto.edu/~hinton/absps/JMLRdropout.pdf)) From what I understood, [this](https://community.deeplearning.ai/t/doubt-about-the-implementation-of-inverted-dropout/4007/2) is the formal justification, but an example may be clearer: \>\>\> m = 1000 \>\>\> keep\_prob=0.5 \>\>\> r = np.random.binomial(1, keep\_prob, m) \>\>\> y = np.random.normal(1, 0.1, m) \>\>\> np.mean(y) 1.0004794606768865 \>\>\> np.mean(r \* y) 0.5123941559332359 \>\>\> np.mean(r \* y / keep\_prob) 1.0247883118664718 Let me know if something doesn’t make … > [@Backward\_Propagation\_With\_Dropout](https://community.deeplearning.ai/t/backward-propagation-with-dropout/8920): > > Hi, While using inverted dropout for regularization, during every iteration of gradient descent we are already dropping out and scaling activations during the forward pass. Why is it needed to explicitly set the switched off dAs to 0 and then scale the dAs in the backward pass? Shouldn’t it automatically happen as a result of the dropouts in the forward pass? My logic: If we implement inverted dropout on AL then shouldn’t dAL already have the corresponding dropped elements zero’d and the rema… --- _[View the full topic](https://community.deeplearning.ai/t/implementation-keep-prob-in-dropout/18832)._