Gradient Check Course 2 Wee1 Ex4

Course Q&A Deep Learning Specialization Improving Deep Neural Networks: Hyperparameter tun
1

Dear all,

I do not really understand the usage of the for loop to go throught the parameters in Grad Check exercise the one to compute de gradapprox. Why do not vectorize this?

for i in range(num_parameters):

    # Compute J_plus[i]. Inputs: "parameters_values, epsilon". Output = "J_plus[i]".
    #(approx. 3 lines)
    # theta_plus =                                        # Step 1
    # theta_plus[i] =                                     # Step 2
    # J_plus[i], _ =

There is something here I do not understand in the instructions, if you copy the parameters to theta_plus, you will copy a (N,1) column vector. but later on to take only the “[i]” value to compute J_plus[i].

Later on, in order to be able to use the function vector_to_dictionary, you need to supply a vector not a scalar

So there is a mess in all this, or we use loops or we vectorize this whole thing so we can use the vector_to_dictionary function and others

Best Regards
Mario

2

Hi, @Mario_RSC.

Good question.

The problem is you have to hold the rest of the parameters constant while you approximate the gradient with respect to one of them. You’re not using theta_plus[i] only to compute J_plus[i]. You’re nudging theta_plus[i], but you’re using all the parameters to approximate the gradient. Does that make sense?

Having said that, there are probably more optimal implementations :slight_smile:

3

I see thanks. I will try just for fun some kind of boolean mask.
Best Regards.

1 Like