I guess I have completely misunderstood what differentiation means with respect to theta.
I had assumed that in exercise 2 the formulas we were coding were J = theta * x for forward propagation
and dtheta = theta / x for the backward propagation.
These work in ex 2 but fail in Ex 3 which uses those helper functions.
Can you advise please where I am going wrong? I just can’t see it.
Ian
That is not the correct code for back prop in the 1D case. And in the N-D case, they implement forward and back prop for you, right?
So I guess I don’t see what the question is.
Hi Paul
It isn’t the coding I am looking for but the correct formula. Differentiation with respect to theta for a real number doesn’t mean much to me. I have reviewed the videos and cannot see what formula I should be using.
Ian
Did you bother to read the instructions for that section of the notebook? They literally wrote it out for you.
Yes those are exactly what I used, and I get “all tests passed” for both forward and backward prop. However when i implement these in the 1D gradient check I get a result of
which is way too small.
As i can’t find anything wrong with the gradient check I can only surmise that I misinterpreted the formulas above.
Why do you believe that is wrong? It is less than the error threshold, right? It’s exactly what I see with code that passes the grader:
Your backward propagation works perfectly fine! difference = 2.919335883291695e-10
The point is that there are no errors in the back propagation code for the 1D case, so your gradient check shows that everything is correct by producing a difference value less than the error threshold. It’s only in the N-dimensional case that they add the intentional errors in back propagation to show that your gradient checking actually works and catches the errors.
I believe it wrong because the output of gradient check is three factors too small. 1e-7 is what i expected at that point.
I may be over=thinking again.
Yes, you are overthinking. I get the same answer. The 10^{-7} is just an upper bound for the error.
Hi guys, im getting even a smaller number for the 1D
Your backward propagation works perfectly fine! difference = 1.221195460051932e-20
the backprop seems too easy to be true.
What I interpretate from 𝑑𝑡ℎ𝑒𝑡𝑎=∂𝐽/∂𝜃=𝑥 is that dtheta = x, although does not make sense to me at all, although all autograder test pass.
Some help is appreciated.
Javier
Your formula for dtheta is correct. This is a highly oversimplified case.
The grader here does pass some incorrect code. I tried the mistakes that I’ve seen in the past that pass the grader, but they don’t give the result you show. It would be interesting to check your code. Please check your DMs for a message from me about how to proceed.
Just to close the loop on the public thread: the issue turns out to be just a misinterpretation of the math notation they use for the norms in the notebook. When they say ||x||_2 to indicate taking the norm, the 2 there is a subscript which means that it’s the “2-norm”. It’s not telling you to square the norm. So anyplace you see that, you can just use np.linalg.norm and it’s all good! 