What is g'(Z) [g prime of Z]?

please take look at image.

G(Z) is the activation(linear function) so gâ€™(Z) should be the first order derivative.

thatâ€™s what A is, right.

this should be dA.
dZ=dA*dA ???

No, sorry, but Prof Ngâ€™s notation is that the dA or dZ values are:

dA^{[l]} = \displaystyle \frac {\partial L}{\partial A^{[l]}}

dZ^{[l]} = \displaystyle \frac {\partial L}{\partial Z^{[l]}}

So youâ€™ve got the derivative â€śupside downâ€ť. Remember that weâ€™re applying the Chain Rule here to compute the overall derivatives of the final scalar cost J w.r.t. whatever the parameter in question is. E.g.:

dW^{[l]]} = \displaystyle \frac {\partial J}{\partial W^{[l]}}

So if we have:

A^{[l]} = g^{[l]}(Z^{[l]})

Then applying the Chain Rule gives us:

dZ^{[l]} = \displaystyle \frac {\partial L}{\partial Z^{[L]}} = \frac {\partial L}{\partial A^{[l]}} \frac {\partial A^{[l]}}{\partial Z^{[l]}}

So that is just the normal mathematical notation for the formula you are asking about when expressed in Prof Ngâ€™s shorthand notation:

dZ^{[l]} = dA^{[l]} * g^{[l]'}(Z^{[l]})

2 Likes