Shallow vs Deep Neural Network

How can we decide whether a NN with 4 layers and 200 neurons each or
20 layers with 2 neurons each work?

Like more neurons in fewer layers or many layers with few neurons in each layer?

Hi there,

there is no right or wrong, but you can think about if the dimensional space which is spanned by the 2 neurons would be sufficient to solve your problem. Finally it’s your task to design a good architecture, also with trial and error.

I have seen it often, that ML engineers rather tend to increase the feature dimensions (or number of neurons) in the first hidden layer compared to the input layer which can be interpreted as giving the net more opportunity to learn complex behaviour.

I think this thread can be interesting for you:

Please let me know if it answers your question.

Best regards

In addition: In this thread the same questions is also discussed, @ajaykumar3456.

Best regards

in my model, i get the best results when I set my NN model architecture to be 12 hidden layers with 120 neurons for each
is it normal to use such a large number?