I just started the Advanced Learning Algorithm course and I am currently on the application of Neural Networks in Face Recognition.
Andrews keeps saying the hidden layers in the network automatically learn or detect patterns in the data and make predictions or produce activations. So I presume that the neurons or models in the hidden layers are technically unsupervised learning models?
Also, without a prerequisite understanding of unsupervised learning algorithms, can one build face recognition models?
The unsupervised algorithms not mean that the model calculate the hidden layers automatically but it mean the the data not labeled like for example linear regression it is data is labeled (x,y) for example [x (4.5)->y(3.6) , x (9.6)->y(1.6)…etc] and you can predict new values … In another hand unsupervised algorithms data is only x [x(3.6) , x(9.6) , …etc] in this case you didn’t predict any new values but you can divide your data in clusters and doing so another algorithm which will you will learn in course 3 …so briefly neural network can’t be considered as unsupervised algorithms it is supervised algorithms as it also predict new values and it is data is labeled (x,y)
please feel free to ask any questions,
Thanks,
Abdelrahman
We could be fooled into thinking that the hidden layers are unsupervised learning models. But there is a catch here - Backpropagation.
We have a cost function J and the cost is evaluated only at the output layer by comparing the predicted value \hat {y} against the target variable y. However, what the learning algorithm really cares about is not J but rather \frac {\partial {J}} {\partial {w}} - If you recall, the update equation is w = w - \alpha . \frac {\partial {J}} {\partial {w}}
So, as long as we are able to find \frac {\partial {J}} {\partial {w}} for every neuron in every layer, the learning algorithm is able to modify each of the weights such that different neurons can be tasked to detect different edges, patterns or features in the image. In the layers near the input, the neurons might detect the simpler edges of the face, while in the layers closer to the output it would detect the larger features of the image such as a nose, ear, mouth etc by putting together the edges detected in the initial layers. As we go even closer towards the output layer, all of the features of the face such as the nose, mouth etc would come together to form the face.
Bottomline - To facilitate the above process, we need a mechanism to find \frac {\partial {J}} {\partial {w}} for every neuron of every layer. A technique called Backpropagation, which is essentially the chain rule in calculus, does precisely this and forms the backbone of the Neural network.