Failing to understand how dJ/dd = 2 in the backward prop based derivation explanation

Course Q&A Machine Learning Specialization Advanced Learning Algorithms
1

Here’s a screenshot of the slide

image
image1738×624 93.8 KB

I understand doing derivatives using Calculus.

So, with J = 0.5*d**2, dJ/dd = 0.5*2*d = d

So, in the forward prop, when d = 2, dJ/dd = 2.

But I didn’t understand the backward prop explanation to get the same answer.

J = 0.5*d**2

  • if d = 1, J = 0.5
  • if d increases by 0.001, d = 1.001. But then J = 0.5*(1.001**2) = 0.501

So, if d increased by 0.001 from 1 to 1.001, J also increased by the same 0.001 (from 0.5 to 0.501).

Why did the slide say that J increased by 0.002? It looks like the 0.5 (or 1/2) in J = 0.5*d**2 got ignored.

2

Replying to myself …

I just realized that I cannot plug in any arbitrary value of d (In my original post, I plugged in a value of 1 to d.)

I need to put in the same value that’s seen in the forward prop (or based on the assumed constants w, b, x, y).

Then the explanation in the slide fits.

  • if d = 2, J = 2
  • if d = 2.001, J = 2.002
  • So if d inc by 0.001, J inc by 0.002
2 Likes