I would think since x.shape = (2,1), z[1] would need dimensions (?, 1)… what am I missing?
Yes, since x has 2 elements, each neuron in layer 1 will have 2 inputs. But this question is about the outputs of layer 1, right? How many neurons are there in layer 1? That’s the thing that will determine the answers.
I am confused. In my notes, the number of notes just matches number of columns in W[1]… it doesn’t determine the shape of z[1].
Well, there are several ways to look at this. The picture you show is a diagram of the network. In that picture, you can see that there are 4 neurons in Layer 1. So each one will be part of the output of layer 1, right? So how many elements will it have?
Or you can look at this in terms of the math formulas for implementing the forward propagation. That would be:
Z1 = W1 \cdot X + b1
and
A1 = g^{[1]}(Z1)
Where g^{[1]}() is the activation function for layer 1. So if X is an input matrix with 2 features and m samples, then X is 2 x m. So in order for the dot product multiply of W1 \cdot X to work, that means W1 needs to have 2 columns to match number of rows of X. So then how many rows does W1 need to have based on the diagram of the network that you showed?
Thank you. I think I was confused because I had a diagram in my notes showing computation of A[1] for single computation node. In reality, there are multiple nodes so X is not only a column vector. In reality, z[1] should have dimensions (?, 4)
The second dimension of Z1 (the number of columns) is the number of samples, right? So that is m. The first dimension is the number of neurons in layer 1.