Neural Networks question

Course Q&A Machine Learning Specialization Advanced Learning Algorithms
1

I’m currently on week 1 of the neural network model section. How do we find out the number of layers in a neural network?

2

There is no formula for that, it generally speaking a trial and error process whichever works best for you dataset, and also it can be similar to models used in similar applications.

There are though automated searching techniques like random search , bayesian search etc.

1 Like
3

Ideally you set the number of hidden layers (and the size of each hidden layer) so that you get good enough performance from the model, but without adding so much complexity that training becomes difficult or consumes too many resources (either CPU cycles or computer memory).

1 Like
4

How many nodes or layers are generally used or start with when we create a model?

5

There are lots of rules of thumb. None of them work in all cases.

Personally, I start with one hidden layer, with the number of units as the square root of the number of input features, or twice the number of output labels (whichever is larger).

I optimize that model, and use it as a reference for exploring the following variations:

  • Increase the number of units in the hidden layer.
  • Then add one more hidden layer.
  • Iterate through this process until you get “good enough” performance.

You don’t need infinite optimization, just find a model that does the job you need.

1 Like
6

Interesting… It’s my first post in the forum and basically the main thing I have been wondering about NN and the related lectures. Either I have missed it somehow or something as important as number or layers and nodes should be made part of the lectures. It’s a pretty basic building block of the modelling.
Following the lectures and examples those parameters/numbers were kinda dropped on us without explaining where they come from.