Hey guys, So i’m curious about hidden units. It shows that if u have more hidden units, then the better the accuracy is. However if it’s too many, then it will cause overfitting. My question is that how is it possible that more hidden units means the more better the accuracy is? Cheers!
Hi, @CourseraFan!
Adding more hidden units increases the complexity of the model. This can make the model to extract more features and usually to get better accuracy (depends on the problem).
However, adding too many layers, especially with a small dataset, can make it learn some characteristics that are not too relevant or data irregularities. Therefore, the training accuracy will still remain high but the validation and test metrics will not.
Hi!
Just to add a little, each neuron in the hidden layers is trained to recognize something different. The more complex the inputs are, the more neurons we may require to get those features that need to be recognized to train the model.
Also note that we initialize the neurons differently to achieve the symmetry breaking and then we let them learn through gradients and back propagation.
Thanks 
1 Like
@Rashmi Absolutely! Thanks for the comment! 