I know that several discussions are already done on this topic but I still don’t understand, so let me ask this question as a new topic. Is scaling the activation units by 1/keep_prob to make the sum of the values of activation units that are retained after dropout the same as the sum of the activa…

Hello @ricky_pii , Thank you for elaborating your question so clearly. The key here is that we do not intend to keep the value exactly the same, and as you have exemplified, keeping it exactly the same is not quite possible. Therefore, the idea is that we hope to make them roughly similar (as in y…

Why do you divide the activations by keep_prob when you use drop

Course Q&A Deep Learning Specialization Improving Deep Neural Networks: Hyperparameter tun

paulinpaloalto May 21, 2023, 4:46pm 4

Hi, Ricky.

It looks like you already found this thread, which has a pretty complete discussion of this point. Did you also read the later replies on that thread? E.g. this one, this one and this one?

Topic		Replies	Views
Dropout scaling fix (division by keep_prob) Improving Deep Neural Networks: Hyperparameter tun	3	678	September 28, 2022
[C2W1 - Regularization] A question about inverted dropout scaling factor Improving Deep Neural Networks: Hyperparameter tun	3	1076	January 27, 2024
Expected Value Stays the Same After Scaling In Dropout Improving Deep Neural Networks: Hyperparameter tun	2	561	October 4, 2021
Week 1 - Doubt in Dropout Regularization lecture video Improving Deep Neural Networks: Hyperparameter tun	7	752	June 16, 2021
Inverted dropout Intuition? Improving Deep Neural Networks: Hyperparameter tun	3	671	May 24, 2022

Why do you divide the activations by keep_prob when you use drop

Related topics