Course 2 week 1 Final Practice Lab
I’m having troubles understanding why 25 and 15 specifically were used for number units in layer 1 and 2 respectfully.
I’d appreciate it if somebody could help me with this…
It’s a natural question. The answer is “from experience”. There are lots of choices that you need to make when you design a neural network to solve a particular problem. How many layers do you need? How many neurons in each layer? Which activation functions to use in the hidden layers? And so forth. We are creating a mathematical function to perform a given task, so we need the function to be complex enough to perform well. But if we make it too complex, then there can be some negative consequences: the training may be more expensive to execute (both in terms of the memory required and the total cpu time and wall clock time) and the resulting model may exhibit what is called “overfitting”, which means that it is too tightly fitted to the training data and does not do as well as we want it to on real input data. So we need a function that is just complex enough, but not too complex.
We are just at the beginning of the learning process here, so please “hold that thought” and learn from all the examples you’ll see as you go through the various courses here. You will see what has worked in lots of examples and will learn techniques for designing and tuning neural networks to solve various kinds of problems.
1 Like
The number of units in neural network layer depends on
Input Layer : Number of neurons in input layer is equal to the number of features in the data. For example a data is defined by 2 features = 2 inputs or by an image of 20 x20 = 400 input.
Output layer : The number of units in output layer depends on whether the model you are creating a classification model or regression model. A classification model can have one or more neurons in the output layer where as regression model has a signal neuron in the output layer.
Hidden Layer : Now this is where your doubt was about the choice of units. The usual way to choose the hidden layer units is making sure it is between the size of the input and output layers. The usual normal is number of hidden unit layer be two-third of the size of the input and the size of the output layer. This is not a mandatory practice but an usual approach based on the data distribution, features, complexity in data statistics which paul was pointing to.
Depending on data complexity : An usual approach(not mandatory), less complex data, 1 to 2 hidden layer is sufficient. More complex data, 3 to 5 hidden layer might be required.
Number of layers in neural network : Usually it is said increasing number of layers is more better approach than increasing the number of neurons in a layer, but sometimes increasing number of layers comes with increasing training time depending on the complexity of data.
Other specific factors are cross-validation data features where input, neuron units and output layer should be incoherent but not similar.
There are other complex terms such as overfitting and underfitting of data depending on how we choose and distribute data between validation, training and testing data.
Regularisation techniques which includes hyperparameters depending on data distribution.These are complex terms which you will understand when you take up Deep Learning Specialisation Course.
Regards
DP
1 Like