I would appreciate a hint on what the bolded part in the dZ2 = np.multiply(dA2, np.int64(A2 > 0)) is doing? The description in the Numpy documentation “Aliases for the signed integer types ( … ) with the specified number of bits.” didn’t make much sense to me. Thanks ahead!
That is g'(Z) if g(Z) is ReLU, right? The formula you are implementing there in full generality is:
dZ^{[l]} = dA^{[l]} * g^{[l]'}(Z^{[l]})
ReLU is 0 for z \leq 0 and z for z > 0, so think about what that implies about its derivative.
Of course the expression A2 > 0 has type Boolean, right? You actually wouldn’t have to explicitly cast it to a numeric type: the normal “type coercion” rules in python would give you a sensible result. But it doesn’t hurt to be explicit about it. If I’d been writing that code, I would have cast it to float64 instead, but whatever …
2 Likes
Thank you, Sir! That’s what I have been looking for!