I made it to this video Machine Learning Specialization - DeepLearning.AI
Now my question is, how the layers are computed down to a scalar values? if they are at all.
Because for me to get a final value to do a comparison against >0.5 for classification, it needs to be a scalar value.
This might be similar The output of a neural network layer
Architecturally, each activation unit has a scalar value.
When implemented, the activation units may be computed as a vector of scalars, with a vector element for each unit in a layer.
“activation unit”? are these the neurons or the layers?
When you design a neural network, you start with the size of the inputs and you know what kind of answer you want. If you know that the purpose of your neural network is to generate a binary (“yes/no”) answer, then you design the layers in such a way that the output of the last layer is a single value. For every layer of the neural network, you know the size of the input and you get to choose what size of output you want to generate. In other words, you get to choose the number of output neurons from each layer and that becomes the input to the next layer. For the output layer, if you want a “yes/no” answer, then there is only one output neuron from the output layer.
Yes. A layer is a collection of neurons.
Matrix algebra.
The number of units in each adjacent layer determines the size of the weight matrix that connects them.
No it was string manipulation between those two Denses.
Of course I know it was matrix algebra. Lol
Now are you able to or can you help us on how it’s done so we the students can understand how a vector of row 3 went from a being a vector to a scaler value for classification?
As I said, it’s matrix algebra.
Read this through “matrix multiplication”.
Think about how matrix multiplication works:
If you multiply an m x k matrix times a k x n matrix, then the result is m x n, right? Now suppose k = 3 and n = 1.
My intention is not to disrespect anyone. I am just calling out lower quality answers.
I could have said the same that I feel disrespected when someone is responding with just anything instead of a proper answer.
May be they are doing this to encourage students to dig deep themselves and if that’s the reason, I apologize. I wasn’t aware of it.
i think it becomes a scalar only at the final output layer. for example, if we have 4 input features, the first hidden layer has 5 neurons, the second hidden layer has 3 neurons, and the output layer has 1 neuron, the input starts as a vector of 4 values. after the computations in the first hidden layer, each of the 5 neurons outputs one value, so we get a vector of 5 values. the same thing happens in the next hidden layer, giving us a vector of 3 values. finally, since the output layer has only one neuron, it produces a single value that we can compare with a threshold like 0.5 for binary classification.
if the output layer had more than one neuron, then it would depend on the task. for multi-label classification, we can compare each output with a threshold. for multi-class classification, we usually use softmax and choose the output with the highest probability instead.
i hope i understood your question correctly.
Thanks to the supporting folks!
Believe me or not, bad intent and bullies do exists in academia! I refuse to bow down to them!
To help visualize what @paulinpaloalto mentioned regarding the dimensions:
If the previous layer outputs a 3 × 1 vector 3 features for a single sample, and the current Dense layer has 1 unit, the weight matrix W for this layer will have shape 1 ×3.
Where:
n^{[l]} is the number of units in the current layer.
n^{[l-1]} is the number of units in the previous layer
So :
Previous layer units: n^{[l-1]} = 3
Current layer units: n^{[l]} = 1
The computation happens as follows:
Substituting the dimensions:
Then applying the activation function a = g(z) keeps it as a scalar (1 × 1), which represents the final prediction/probability.
Hope this step-by-step dimensional breakdown makes the matrix multiplication clear!
