Here’s another thread with links to material about the derivations of the backprop formulas. It also points to this thread about matrix calculus in general that is helpful. As you’d expect, matrix calculus is based on the same principles as univariate calculus, but things get more complicated with more dimensions beyond even the normal partial derivative notions.

1 Like