Derivation of DL/dz

Course Q&A Deep Learning Specialization Neural Networks and Deep Learning
21

I leave some Python cents here. It is just one hidden layer back-pass.

    # First, compute the new predictions `y_pred`
    z_2 = np.dot(x_mat, W_1)
    a_2 = sigmoid(z_2)
    z_3 = np.dot(a_2, W_2)
    y_pred = sigmoid(z_3).reshape((len(x_mat),))
    # Now compute the gradient
    J_z_3_grad = y_pred-y # Trick: This is the gradient of the log loss function: https://community.deeplearning.ai/t/derivation-of-dl-dz/165
    J_W_2_grad = np.dot(J_z_3_grad, a_2)
    a_2_z_2_grad = sigmoid(z_2)*(1-sigmoid(z_2))
    J_W_1_grad = (np.dot((J_z_3_grad).reshape(-1,1), W_2.reshape(-1,1).T)*a_2_z_2_grad).T.dot(x_mat).T
    gradient = (J_W_1_grad, J_W_2_grad)
9 Likes