How are the number of layers and number of neurons in each layer of a neural network determined?
By using experiments to find the simplest system that gives good enough results.
Its a trial and error process. In case of very large networks, we can look at tried and tested network topology as a reference.
Hi, @Siddharth_BH !
Although it’s true that this process is all about trial and error, you can always use some popular backbones as a starting point, like VGG (although a bit old-fashioned), ResNet, DenseNet and EfficientNet
2 Likes
hi there so the number of neuron in for example the first hidden layer has no relation with input parameters?? same question for hidden layers.
Thanks.
Pablo
1 Like
Hello Pablo, no, there is no such relation that you have to follow when building a NN. You are free to use any number of neruons and any number of layers but at the end of the day, you want to experiment different architectures to find the one with the best cv score.
However, if I were to build a NN, I probably will start my first hidden layer with more neurons than the number of features in my input X, because we just need to give more freedom for the NN to try to find some good combinations of my original features!
Raymond
thanks raymond for your comments. it seems right.
Thanks
Pablo
It is common to pick a structure whose first hidden layer has comparable size to the input and the sizes gradually decrease to the size of the output layer. This is not necessarily the best way to do things, but it usually works reasonably well. It really is trial and error. Only the sizes of the input and output layer is determined by the dataset. The rest is up to you.