Relationship between Propagation (forward and backward) and derivatives in the code

[Niranj_Kumar]

So, based on the 1st required assignment for the first course (Neural Networks and Deep Learning), I’m attempting to put the pieces together. When writing the code, I have to use derivatives (calculus-based) in the propagation section and optimization sections. Is it for simplification purposes? Or does it depend on the function (sigmoid, linear, etc.)?

[TMosh]

Which course are you attending?
Please compare that with the name of the Forum topic where you posted your question.

[paulinpaloalto]

@Niranj_Kumar

Is it for simplification purposes? Or does it depend on the function (sigmoid, linear, etc.)?

I’m not sure what you mean by “simplification” there. Yes, the functions matter. Backward propagation is driven by the derivatives of the forward propagation functions. Of course the derivative of sigmoid is different than the derivative of relu or tanh, so you have to be cognizant of all the functions used in forward propagation. Then it’s just a big application of the Chain Rule.