Prof Ng has specifically designed these courses so that they do not require the students to know any calculus (even univariate calculus, let alone matrix calculus), so he does not cover the derivations of a lot of the formulas which involve calculus. If you would like to dig deeper and have the math background, there are lots of resources available. Here’s a local thread with a bibliography, which includes text books that cover the actual math behind all this. One book that is more math oriented is Goodfellow et al, which is listed there.
Here are some good websites that will also cover the derivations of back propagation:
Here’s a website from Cornell that covers the derivation.
Here’s a good introduction to the matrix calculus you need in order to follow the above.
The Matrix Cookbook from Univ of Waterloo is also a valuable resource for general Linear Algebra topics as well as matrix calculus.
Here are some notes from Stanford CS231n that give a good overview and insights on back propagation.
Here’s a bit deeper dive on the math also from Stanford CS231n.
Here are notes from EECS 442 at Univ of Michigan.
Mentor Jonas Slalin also covers all this and more on his website. That’s just the first page in his series.