is there a specific papers to read to understand the neural network under the hood ? i mean i got an intuition but i want to understand how the layers learn from each other i mean it is just math and i know of course that we try to minimize the loss so that we could tune the weights and then we try to generalize the model but i want to know hoe the layers outputs is referring to some other features they learned from their inputs without knowing what they are , i just want to know all those details not just an intuition
Hi, welcome to the community.
The mathematical mechanism by which the layers learn is quite well understood: forward propagation produces activations, the loss measures the error, and backpropagation uses the chain rule to calculate how each parameter contributed to that error. Gradient-based optimization then updates those parameters.
What becomes much less straightforward is understanding what the representations learned by individual neurons or layers actually mean. A network is not explicitly told, for example, “this neuron should detect edges” or “this layer should represent faces.” These representations emerge from optimization, and a feature may be distributed across many neurons rather than corresponding neatly to one neuron. Understanding and interpreting these learned representations is still an active area of research.
If you want to explore that level of detail, I think the Distill Circuits work is an excellent resource:
It explores how learned features and circuits arise inside neural networks and should take you beyond the usual high-level explanation of forward propagation and backpropagation.
It’s just math. There is no real intuition happening. Only the appearance.
Hi @Abdelrahman18 , and welcome to the community!
To add a little visual intuition to the excellent points made by @lukmanaj and @TMosh, I like to think of a neural network as a massive audio mixing board with millions of sliders (these are the weights and biases).
Initially you will see that all the sliders are set randomly and so the output is just noise so while every time data passes through, you see the math (Backpropagation and Gradient Descent) tells us exactly which sliders to nudge up or down to make the output sound a tiny bit better (minimizing the loss) what i mean.
As @TMosh rightly pointed out, there is no conscious “understanding” happening here actually , the mixing board doesn’t know what music is really!!
butt it’s purely calculus and matrix multiplication automatically adjusting the sliders.
However, as @Lukmanaj beautifully highlighted the magic happens in the results of that math!.
that’s mean If you examine specific clusters of sliders after training, you might find that some have perfectly organized themselves to act as a “bass booster” or a “vocal filter” (feature representations).
So you see the math blindly optimizes! , but the resulting structure mimics intuition!
you can look at this :
Enjoy your journey in the course!