Week 3 Quiz - Potential Contradiction

Course Q&A Deep Learning Specialization Neural Networks and Deep Learning
1

Hi all!

The week 3 quiz, Shallow Neural Networks, suggests that W^ [1] is a matrix whose columns are the parameter vectors. This makes sense, as we typically matrix multiply W^ [1]T by the inputs to each neuron, where the inputs are a column vector.

The confusion I’m facing comes from another question on the quiz which says that the number of rows in W^ [k] is the number of neurons in layer k, and that the number of columns is the number of inputs to the layer. Doesn’t this mean that the rows are the parameter vectors in W^ [k]? Since each row represents a different neuron, that row must be the parameters for that neuron(?), suggesting that rows are now the parameter vectors, and not the columns. This seems to contradict the other question saying that the columns are the parameter vectors. Is this not right?

2

The dimension of W^{[1]} is n^{[1]} x n_x, where n^{[1]} is the number of output neurons in layer 1 and n_x is the number of features in each input vector. So the number of columns is the number of inputs, which (for a layer beyond layer 1) would be the number of outputs from the previous layer.

3

Cheers for this, @paulinpaloalto!

In this case, it sounds like the rows would be the weights for the features since the number of columns of W[1] matches the number of features in the input vector. The troubling bit I found in the quiz was this correction saying that the columns of W[1] are the parameter vectors, and not the rows:
image

4

I am not sure what they mean by that. I can think of two theories:

  1. They really mean the “feature vectors” in the W^{[1]} case. Those are legitimately column vectors.
  2. They mean it in the sense explained in the lectures where Prof Ng treats the rows of W as the transpose of the w weight vectors as we had them in Logistic Regression. So they mean the w vectors before the transpose.

If it is theory 2), then it is explained on this thread.

In either case, I think that way of explaining their answer is unnecessarily confusing. I will research more and then file a bug asking them to make this more clear.

Thanks for pointing this out.

2 Likes