Week 3 (W3) - W matrix values

Course Q&A Deep Learning Specialization Neural Networks and Deep Learning
1

Hi everyone!
I have a question on the values inside the matrix W. From what I understood from the video (link in the bottom) those values are different in each row. What I don’t get is that shouldn’t those values be the same for each training example? If not what you are doing with the vectorization is changing the neuron shape. Before vectorization A[0] had a length of nx (nx being the length of each vector of each training example). Now it appears that A[0] has a length of nx times m (m bieng the number of training examples) to respect the original length the weights of matrix W should be the same in each row right? I’m confused about this.
Thanks in advance.

Nico

2

Yes, the rows of the W matrix represent the coefficients for a given output neuron at that layer. So they will be different per row. But they are the same for every input sample, right? Think about what happens at the first layer when you compute:

Z^{[1]} = W^{[1]} \cdot X + b^{[1]}

The matrix W^{[1]} has shape n^{[1]} x n_x, where n^{[1]} is the number of output neurons for layer 1 and n_x is the number of input features in each sample.

Then X has dimensions n_x x m where m is the number of input samples in the current batch of inputs.

So think about what will happen in that dot product W^{[1]} \cdot X: each row gets “dotted” with each sample (each column of X). The result will be that Z^{[1]} has shape n^{[1]} x m.

1 Like
3

Here’s a thread from a while ago that also talks in some detail about this material in Week 3 and how the W matrices are organized and how they work.

4

Thank you so much! I got confused with the image. the one from the class and the thread you posted. In there I saw X1 X2 X3 with different wights so I thought they were different weight for different training example, but they are the same training example, just the components of one training example right?

5

Weights apply to each feature, not each example.

6

Yes, in the screenshot on that thread I linked, the values x_1, x_2 and x_3 are the features (components) of one sample value x. That would be one column of the matrix X in my notation in my post on this thread.

1 Like