[C2_W1_Lab02_CoffeeRoasting_TF]: [Week#1]: Not clear on the dimensions of weights in the second layer

Course Q&A Machine Learning Specialization Advanced Learning Algorithms
1

Consider the below statement from the optional lab [C2_W1_Lab02_CoffeeRoasting_TF]:

"Let’s examine the weights and biases Tensorflow has instantiated. The weights π‘Š
should be of size (number of features in input, number of units in the layer) while the bias 𝑏 size should match the number of units in the layer:

  • In the first layer with 3 units, we expect W to have a size of (2,3) and 𝑏
    should have 3 elements.

  • In the second layer with 1 unit, we expect W to have a size of (3,1) and 𝑏
    should have 1 element."

In the first layer, there are 3 neurons and the input X has 2 features (Temperature and Duration). So, this makes sense for first layer, where 3 neurons with 2 feature each for weight and 3 values for bias, which is 3 * 2 + 3 => 9

However, in the second layer, where there is only one neuron, not sure how the number of features became 3 here (as it was 2 in the first layer).

Please explain more on this.

2

One way to look at this (there is more than one, because there really isn’t a universal standard) is that each adjacent pair of layers of units is connected by a weight matrix.

The dimensions of each matrix are (outputs x inputs), as the rows and columns, which refers to the number of units in the adjacent layers.

So the number of rows in the 1st weight matrix is the same as the number of columns in the 2nd weight matrix.

If you draw a little pencil sketch, it is easier to visualize.

3

Ok, got it. To convert the 3x1 output from first layer to 1x1 at the second layer output, this dimension of 1x3 makes sense.
If I understand it properly, after the first layer output, it can no longer be mapped directly to the features (temperature and time) that were originally input to the first layer, rather it might be some derivates of it?

4

Yes, that is correct.

5

Thank you, for the clarification.