Neural Network Architecture - What factors in determining # of layers and units/nodes in each layer

AI Discussions
1

I am curious if anyone has any insight into how to determine the number of hidden layers and number of units/nodes in each layer of Neural Network, even if this is not an exact science or determined via formula.

I am currently taking ML Specialization, and in course 3, Unsupervised Learning, there is an example of Lunar Lander using Reinforcement Learning algo with NN algo embedded where by the network architecture consists of 12 Inputs in Input Layer, 1 output in output layer (both of these are understandable), but chooses to have 2 hidden layers each consisting of 64 units. There is no mention as to how these numbers of layers and number of units in each layer were selected.

Thank you!

2

It’s by experimentation.

The goals are:

  • Create a complex enough model that you get sufficiently good results.
  • Avoid creating an overly complex model that is too difficult/expensive/time-consuming to train and use for predictions.
3

It’s an important question, but as Tom says there is no straightforward or easy answer. Since you are currently taking MLS, the next step would be to take DLS after you finish MLS. DLS is a more advanced specialization that is specific to deep neural networks and Prof Ng will spend a lot of time talking about how to structure your networks, how to evaluate their performance and how to improve that performance in a systematic way. Those will be major topics of DLS Course 2 and Course 3 and then more specific advice for ConvNets in Course 4 and Recurrent Nets in Course 5.