In the above video, Andrew mentioned that earlier layers in a neural network tend to learn simple features like “oriented edges” and later layers learn complex features like part of the face and full faces etc. My question is since we know (at least that’s my understanding) that these neural nets operate in a sequential manner and output of one layer becomes input of the next layers, then how come out of edges (output of first layers) the neural net can detect faces in the next layers? are we passing the input image to all the layers along with previous layers input or what?
I think one way to think about it is that the faces are made up of small edges. The first layers detect small edges, and then later layers try to group those edges together to try and detect more complex features like faces.
When you have a neural network you pass your data through layers of the neuron, a neuron depends on the previous output, saying that learn things like edges is a simplification on what is actually happening, that the neuron convers the input into a vectorize form of output that will be pass to the next neuron, so everything is information of the data that goes to the neuron, also remember you pass your data several times and creates multiple inputs and outputs.
They don’t actually work in stages. Andrew was just giving an intuitive explanation to help understanding the process. The entire NN is updated once on each training epoch.
Think about this as a picture of a point on time, but in reality you iterate over this several times passing your data multiple times in the neural network to update the weights of the model to minimize the error
Now this other picture shows how the accuracy and loss change with each iteration, in every point in time you are passing the data and creating weights and biases for the neural net after you find the best combination with your current settings.
Note that in a well-tuned model, the validation will not degrade with increasing epochs. The results for both Training and Validation should reach a stable plateau.
