I would appreciate a hint on what the bolded part in the dZ2 = np.multiply(dA2, **np.int64(A2 > 0)**) is doing? The description in the Numpy documentation *“Aliases for the signed integer types ( … ) with the specified number of bits.”* didn’t make much sense to me. Thanks ahead!

That is g'(Z) if g(Z) is ReLU, right? The formula you are implementing there in full generality is:

dZ^{[l]} = dA^{[l]} * g^{[l]'}(Z^{[l]})

ReLU is 0 for z \leq 0 and z for z > 0, so think about what that implies about its derivative.

Of course the expression A2 > 0 has type Boolean, right? You actually wouldn’t have to explicitly cast it to a numeric type: the normal “type coercion” rules in python would give you a sensible result. But it doesn’t hurt to be explicit about it. If I’d been writing that code, I would have cast it to *float64* instead, but whatever …

2 Likes

Thank you, Sir! That’s what I have been looking for!