Lecture mentions Microsofts 152 layer DNN as an example of very deep NN. In general what is the number of layers to be considered as very deep NN (if there is such a heuristic)?
HI @dds
Deep NN number of layers start with More than three layers (including input and output) qualifies as “deep” learning .I think there isn’t rule that control number of layers of very Deep NN but I think to say it is very Deep NN not depending only on number of layers but also number of neurons of every layer and how many inputs (feature) do you have as if you have small number of features and you implement it on very Deep NN it is very chance to suffer from overfitting
Please feel free to ask any questions,
Thanks,
Abdelrahman
I understand that there isn’t a rule and one would not make sense. I was just wondering if there was a prevalent belief in DL community that, say layers more than 25 is very deep. Or is “very” adjective used when the exhibts certainhaviors such as exploding/vanishing gradient (which can be tested empirically). I do understand that the label of “very deep” would be interesting only if there were certain different tools or methods used for very deep vs just deep.