First lesson, 4 questions about NN

[paulinpaloalto]

@Alireza_Saei

All neurons in a layer receive the same input vector but each one has its own set of weights and bias. During training, the backpropagation algorithm adjusts these weights differently for each neuron based on the error gradients they contribute to. This means neurons learn to focus on different combinations of inputs or detect different features.

It might also be worth going one level deeper on that point. The reason this happens is that we must start back propagation by randomly initializing all the weight values. That is a technique called “Symmetry Breaking”. If you didn’t do that and started all the neurons with the same weights, then they would all learn the same thing and your network would be basically useless because each layer has the equivalent of only one unique neuron. Here’s a thread which discusses Symmetry Breaking at bit more.