I see in the video week 2, backprop video 2 that andrew said the derivative dj/dd = 2. That is when d goes up by 0.001, j goes up by 0.002. I think of this as 2d, however, when I differentiate the equation, it is equal to d. What is happening here?
Please share a screenshot and link to the video.
No, it’s a 2ε, where ε is the amount you nudge the variable d by.
That’s correct. In that video, d was equal to 2 in forward prop, so the derivative is simply \frac {\partial J}{\partial d} = 2 .
1 Like
Any idea how can I find a mathematical equation that would return these answers so I don’t have to substitute numbers and find it out this way?
Some calculus can be used to find those answers, but you do not need to know calculus to complete this specialization.
Actually I have taken some calculus courses. Granted I am a bit rusty on the concepts, but it is incredibly frustrating to not be able to derive a proper equation, which is why I asked here.
I understand, but that was just an example to illustrate how backpropagation works in frameworks like Tensorflow.
Tensorflow automatically computes derivatives for you, which means you don’t have to manually derive such equations.