C2_W1 Question on Forward prop in neural network not clear

Course Q&A Machine Learning Specialization Advanced Learning Algorithms
1

I don’t understand the concept of the question asked. Can someone help with an explanation. Thanks

image
image1392×921 88.5 KB

2

In neural networks, when implementing forward propagation, we calculate the output (also known as activation) for each neuron (or unit) in a layer by multiplying the input vector by the weights and adding the bias for each neuron. The question asks where to place the weights (W) for each neuron when constructing a neural network in NumPy. The weight matrix W is a 2D array where each element represents the connection strength between input features and neurons in the current layer. It gives two options:

  • Rows in W: This implies that each row in the weight matrix corresponds to a particular input feature and its connection to all neurons.
  • Columns in W: This implies that each column in the weight matrix corresponds to a specific neuron and its weights for all input features.

See the figure:

  • The weight matrix W is defined as a 2x3 matrix. Each column of W corresponds to a single neuron:

    • Neuron 1 has weights [1, 2]
    • Neuron 2 has weights [-3, 4]
    • Neuron 3 has weights [5, -6]

    These weights determine how each neuron interacts with the input vector a_in (which is [-2, 4]).

  • The forward pass involves multiplying the input vector with these weights for each neuron and adding the respective biases b to get the output activations.

In the matrix multiplication process used during forward propagation (np.dot(W, a_in) in the code), each column of W contains the weights corresponding to a particular neuron. When you perform matrix multiplication, you multiply each input vector (a_in) by the weights of each neuron (which are in the columns of W). This gives the activation for each neuron. This setup allows the matrix multiplication operation (np.dot(W, a_in)) to calculate the output for each neuron effectively:

  • Columns of W: Each column contains the weights for one neuron, i.e. the dot product np.dot(W, a_in) calculates the weighted sum for each neuron based on the input.
  • Bias: The biases are added to the weighted sum to produce the final activation.
  • Activation function: An activation function (g) is applied to get the neuron’s output a_out.
3

@nadtriana
Thanks. I get it now

1 Like