Understanding Dropout Lecture Video

Hi Sir,

What does it mean in the below bold highlighted statement? Can u please me to understand ?

  1. I didn’t introduce,welcome to my code during drop out because you need other ways, I guess, but not plotting these figures to make sure that your code is working,the gradient descent is working even with drop out. So with that there’s still a few more regularization techniques that were feel knowing.Let’s talk about a few more such techniques in the next video. What does it mean here but not plotting…even with drop out cannot undertand what proff meaning ?

  2. One other alternative might be to have some layers where you apply dropout and
    some layers where you don’t apply drop out and then just have one hyper parameter which is a key prop for the layers for which you do apply drop out and before we wrap up just a couple implantation all tips . { This is the content from lecture vide ]

Should we keep only one hyper parameter like one keepprob only for multiple layers that we are going to apply drop out ? or one keepprob for all nodes in the single layer ?

Hi, @Anbu.

Should we keep only one hyper parameter like one keepprob only for multiple layers that we are going to apply drop out ? or one keepprob for all nodes in the single layer ?

There is no right or wrong answer. If you want to fine tune Dropout for each layer, set different values for keep_prob at the expense of having more hyperparameters to search for. If you would rather have less hyperparameters, define a single keep_prob for all the layers where you apply Dropout.

Regarding the first question, I think it means the loss plot is less reliable since Dropout is effectively introducing noise, but I could be wrong.

Let me know if that helped :slight_smile:

Yes Sir loss plot is less reliable. So Proff andrew what try to mean other ways instead of not plotting figures to make sure gradient descen working even with dropout?

I don’t think he’s referring to any specific method, if that’s what you’re asking. But turning off Dropout, running your code and making sure the loss is decreasing monotonically, as suggested in the lecture, may be enough to test your implementation :slight_smile:

1 Like