Before you post, please make sure to:
Add a clear title Matrix size Confusion
Add all required tags in the tag option (top-right of the post):
dl-ai learning platform
Shallow Neural Networks
Write a detailed description of your issue.
How is the size of the given matrix W is 4 by 3 ? It should be 4 by 1. Also, why we are defining the weights per node, I thought that every input has its own weight, for example x1 should have its own weight w1, x2 should have w2 and so on.
This may be a confusion on the weight matrix numbering.
Looking just at the NN graphic:
There is a W matrix that connects adjacent pair of layers. Each arrow on that graphic is a single real weight value.
The W matrix between the inputs and the hidden layer, with all of the crossing arrows, that’s W1. It’s size is 4 x 3 (four hidden layer units, three input units)
The W matrix between the hidden layer and the output, that’s actually a 4-element vector (because there is one weight value for each hidden layer unit). That’s W2.
So it would be right to say that W11 is a vector consisting of w1,1 , w1,2 , w1,3 and w1,4 , denoting all the wieghts coming out from the input x1 to all the four nodes in the first layer?
This question has come up before. Here’s an ancient scroll that was recently excavated that is worth a look. 
