Dropout initialization

@bahadir
why do we use rand to initialize dropout vector and randn for the weights?

Hi @kunal,

In which course or assignment you have seen this?
While I don’t see an obvious reason for this, I’ll check the course or assignment to be sure, after you inform me.

@bahadir
Course 2 week 1

@kunal,

The initialization notebook in Week 1 has this note which explains the reason: When used for weight initialization, randn() helps most of the weights to Avoid being close to the extremes, allocating most of them in the center of the range.

In dropout, we don’t have that concern and the uniform distribution of numbers between 0 and 1 helps different neurons to be dropped in each iteration. If we use randn() the neurons close to the beginning and end will not be dropped as much as the center neurons.