W2_A2_Calculation of Partial derivatives

Hey @saifkhanengr,
Although not related to your query, I believe that your first equation has one missing term. According to me, it should be as follows:


\frac{dL}{dZ1} = \frac{dL}{dA3} \times \frac{dA3}{dZ3}\times \frac{dZ3}{dA2}\times \frac{dA2}{dZ2}\times \frac{dZ2}{dA1} \times \frac{dA1}{dZ1}


\frac{dL}{dW1} = \frac{dL}{dA3} \times \frac{dA3}{dZ3}\times \frac{dZ3}{dA2}\times \frac{dA2}{dZ2}\times \frac{dZ2}{dA1}\times \frac{dA1}{dZ1}\times\frac{dZ1}{dW1}


\frac{dL}{db1} = \frac{dL}{dA3} \times \frac{dA3}{dZ3}\times \frac{dZ3}{dA2}\times \frac{dA2}{dZ2}\times \frac{dZ2}{dA1}\times \frac{dA1}{dZ1}\times\frac{dZ1}{db1}


And as for your query, you are correct indeed. For computing dW_1, we do not take the derivative wrt any other weights or biases. Same goes for db_1. The reason is simple too.

Consider Z_2 = W_2^T A_1 + b_2. For computing \frac{dL}{dW1}, we need \frac{dZ1}{dW1}, and since from Z1, we get A1, we also need \frac{dA1}{dZ1}. Now, here if we take \frac{dZ2}{dW2} instead of \frac{dZ2}{dA1}, there will be a derivative mismatch, and the back-propagation won’t work. I hope this resolves your issue.

Cheers,
Elemento

3 Likes