Which layer to use dropout regularization

Hello All,

I would like to ask when the dropout should be applied in a CNN structure. Can it be applied for between convolutional neural networks or only for fully connected layer?

Thanks

Yes, you can use dropout both on Conv layers and on FC layers. There are no hard and fast rules for where and when you do that. You’ll see examples in DLS Course 4 of the use of dropout with Conv layers, e.g. in the Image Segmentation with U-Net assignment in C4 W3.

1 Like

Hi @Yagmur_Gulec,

Just like Paul’s suggestion, I recommend you to look for some existing or SOTA architectures and see where they used dropout. Course 4 will give you a lot of names of those architectures to begin your search which hopfully will bring you to some Tensorflow Models such that you can print their model.summary() to find out where those drop-outs are.

For example, check this efficientnet out. Do model.summary(), and search for dropout in the printed summary, and you will find them. If you spend a day, you probably would be able to go through most well-known architectures and know first-hand how people use it.

Good luck!
Raymond

1 Like

And perhaps equally importantly, for each dropout layer you find, check out its rate to see how people set it.

1 Like

Even better if you keep a notebook for your finding, such as the code to recreate the summary for each model, and some links of reference to where you found the model, and any other notes.

You can refer back to this notebook in the future if you have other questions, so that next time you would not need to spend a day looking for reference. If you make a good notebook, I would also recommend you to upload it to your Github and share it on, for example, your LinkedIn so that you network can benefit from it :wink:

Cheers,
Raymond

1 Like