Shallow vs Deep Neural Network

Course Q&A Machine Learning Specialization Advanced Learning Algorithms
1

How can we decide whether a NN with 4 layers and 200 neurons each or
20 layers with 2 neurons each work?

Like more neurons in fewer layers or many layers with few neurons in each layer?

2

Hi there,

there is no right or wrong, but you can think about if the dimensional space which is spanned by the 2 neurons would be sufficient to solve your problem. Finally it’s your task to design a good architecture, also with trial and error.

I have seen it often, that ML engineers rather tend to increase the feature dimensions (or number of neurons) in the first hidden layer compared to the input layer which can be interpreted as giving the net more opportunity to learn complex behaviour.

I think this thread can be interesting for you:

Please let me know if it answers your question.

Best regards
Christian

3

In addition: In this thread the same questions is also discussed, @ajaykumar3456.

Best regards
Christian

4

in my model, i get the best results when I set my NN model architecture to be 12 hidden layers with 120 neurons for each
is it normal to use such a large number?