Course 2 -- Week 1 -- Dropout

[kampamocha]

Hi @Zoooeee,

this question has been asked before, maybe the following posts can be useful:

[C2W1 - Regularization] A question about inverted dropout scaling factor

(this was previously posted on the forum, then moved here) W​hen using inverted dropout, we follow these steps: 1​. Compute dropout matrix D D = matrix of same shape as A, with zeros and ones (where ones have keep_prob probability) 2. Update Activation matrix by zeroing some elements A = A[Where D1 is 1, else 0] 3. Rescale the activation matrix A = A / keep_prob M​y question: would it not be more accurate to divide by the actual “activation reduce factor for this layer”; i.e i…

Doubt about the implementation of inverted dropout

During the week 1 lecture series where professor andrew introduces the concept of regularization, he talks about using “Dropout” as a technique for the same. Now, while implementing dropout at any layer “l”, we first calculate a dropout matrix for that layer and then multiply the same to A[l] (activations of that layer) to zero out the effect of certain neurons. Once, we are through with this, professor andrew also mentions the significance of rescaling the A[l] matrix by performing element wise…

Hope that helps.