I’m debugging my code for gradient_check_n and I’m looking at calculations at each step.

What I’ve found is a difference between `grad`

and ```gradapprox `` values - but not for all records. For some this is almost perfectly aligned but for some there is a difference of scale 2. See below:

What might be the reason for such a behaviour?

Each row of `gradapprox`

is being calculated as:

`gradapprox[i] = (J_plus[i] - J_minus[i])/(2*epsilon)`

I know there should be an error but the expected value of the check is wrong. What I get is:

**Or** is it an error in the difference calculations?

I’ve never looked at the detailed elements of the *grad* and *gradapprox*. So it looks like you have not corrected the intentional mistakes they put in backprop, but that is fine. The goal is to get one of the two correct answers (the one with the error or the one with the errors fixed).

So this says that there is something else wrong with your calculations. Have another close look at the earlier code before the `gradapprox[i]`

line you show, which looks correct to me.

Just out of curiosity, I added similar instrumentation to my code and here’s what I get:

```
32: gradapprox[i] = [0.], grad[i] = [0.]
33: gradapprox[i] = [0.], grad[i] = [0.]
34: gradapprox[i] = [0.], grad[i] = [0.]
35: gradapprox[i] = [0.9158017], grad[i] = [1.8316033]
36: gradapprox[i] = [0.02451541], grad[i] = [0.04903095]
37: gradapprox[i] = [-0.10797954], grad[i] = [-0.21595908]
38: gradapprox[i] = [0.90281879], grad[i] = [1.80563782]
39: gradapprox[i] = [0.], grad[i] = [0.]
40: gradapprox[i] = [0.], grad[i] = [0.]
41: gradapprox[i] = [0.], grad[i] = [0.]
42: gradapprox[i] = [0.19763344], grad[i] = [0.19763343]
43: gradapprox[i] = [0.], grad[i] = [0.]
44: gradapprox[i] = [0.], grad[i] = [0.]
45: gradapprox[i] = [2.24404227], grad[i] = [2.24404238]
46: gradapprox[i] = [0.21225742], grad[i] = [0.21225753]
numerator = 2.3225406048733928
denominator = 8.146602433873603
There is a mistake in the backward propagation! difference = 0.2850931567761623
```

Interestingly it looks like my values agree with yours to the resolution you show for the individual `gradapprox[i]`

values. It is the `denominator`

that is wrong in your case. There aren’t very many moving parts there.

1 Like

Thank you,

indeed, there was a typo in `denominator`

code.