What is g'(Z) [g prime of Z]?


please take look at image.

G(Z) is the activation(linear function) so g’(Z) should be the first order derivative.

that’s what A is, right.

this should be dA.
dZ=dA*dA ???

No, sorry, but Prof Ng’s notation is that the dA or dZ values are:

dA^{[l]} = \displaystyle \frac {\partial L}{\partial A^{[l]}}

dZ^{[l]} = \displaystyle \frac {\partial L}{\partial Z^{[l]}}

So you’ve got the derivative “upside down”. Remember that we’re applying the Chain Rule here to compute the overall derivatives of the final scalar cost J w.r.t. whatever the parameter in question is. E.g.:

dW^{[l]]} = \displaystyle \frac {\partial J}{\partial W^{[l]}}

So if we have:

A^{[l]} = g^{[l]}(Z^{[l]})

Then applying the Chain Rule gives us:

dZ^{[l]} = \displaystyle \frac {\partial L}{\partial Z^{[L]}} = \frac {\partial L}{\partial A^{[l]}} \frac {\partial A^{[l]}}{\partial Z^{[l]}}

So that is just the normal mathematical notation for the formula you are asking about when expressed in Prof Ng’s shorthand notation:

dZ^{[l]} = dA^{[l]} * g^{[l]'}(Z^{[l]})

2 Likes