Multiple neurons in a single layer

Course Q&A Machine Learning Specialization Advanced Learning Algorithms
1

What is the need of multiple neurons in a single layer in neural networks?

2 Likes
2

Each unit in a layer will learn some different characteristic of the data set that helps lead to minimizing the cost.
If the data set is simple, one unit might be enough.
If the data set has complex relationships between the features, multiple units may be needed.

1 Like
3

Hi @gigaGPT great question! As was pointed out by @TMosh several neurons can help to detect different relationships between your variable and the output, this increases your chances to detect useful parameters and select the weights needed to have good predictions. The power of NN came from combining different results into a single prediction, so a high amount of neurons in each layer can help you. Of course, it has its trade-off, if you increase too much the number of neurons you might overfit your data.

I hope this helps

3 Likes
5

That’s because the weights are different for each unit.

6

As per my research knowledge, each unit (neuron) in the hidden layer is used to compute a particular scenario/characteristic in an image (i.e. a particular block of pixels).

For example: if we train our neural network with 3 hidden layers, each layer consisting of multiple neurons. Maybe here first layer is computing the pixels corresponding to edges/lines in an image. But the presence of multiple neurons in firs layer maybe present because of following:

  • First Neuron: Maybe computing only Horizontal Edge pixels.

  • Second Neuron: Maybe computing only Vertical Edge pixels and so on.

7

Thanks. How are the weights specified? how does this impact the number of neurons desirable for the current layer? I’m working through the Advanced Algorithms course (now on week 3) but there seems to be very little said about how individual neurons operate.

8

The rule of thumb is to initialize the weights randomly.

9

Thanks. Info such as this seems to have been skimmed (or skipped) over in the Advanced Algorithms course. Can you tell me where to find this in that course or perhaps another? I have more questions on this neuron / layer concept.

10

You can take the deep learning specialization offered by DeepLearning.AI after finishing this one.

11

Okay I will do that. Thanks so much!