Hi, when we first create the logistic classification, I understood that each parameter contributes to the Z, then a and finally the cost function. However, how come each node in the hidden layer will be different if each one is the same combination of input parameters? but this time repeated by the number of nodes ?
I have this same question. Each node seems to take same input and have same output, so I also don’t understand how the results would be different. Just finished the third video of week3.
It is a good question: the reason is that you start with the initial values of the weights as different for every node. This is called “Symmetry Breaking” and Prof Ng does talk about it in the lectures. You’re right that if you started out with all the weights the same, then the outputs would be the same and back propagation would be the same, so you basically end up with only one real neuron effectively.
Here’s a thread which talks more about Symmetry Breaking.
2 Likes