Error in Adam implementation

Hi,
I’m getting slightly off values for all the results and I don’t see where is the error.
I checked the formulas several times.
W1 =
[[ 1.63949493 -0.62691477 -0.54326465]
[-1.08769515 0.85031501 -2.28657079]]
W2 =
[[ 0.33262355 -0.26414959 1.47708248]
[-2.0457142 -0.30744639 -0.36898502]
[ 1.14872646 -1.09849003 -0.15727519]]
b1 =
[[ 1.7549895]
[-0.7461017]]
b2 =
[[-0.89102966]
[ 0.02699863]
[ 0.56780324]]

Should be a python code issue?
(beta1)**t ?
np.square(grads[‘dW’ + str(l)])?

I just found the error myself. I thought the epsilon was inside the square. Not outside.
image

Now it’s fine.

2 Likes