Can somebody help me with error in adam algorithm!?

[Branko_Trifkovic]

When I run my implementation of adam optimization, got this error (s_db_correct is negative value). And I dont know where is a problem
Here is implementation of adam algorithm

And this is function where I compute gradients for w and b

[TMosh]

Your code in update() seems to rely on the value of s_db_correct always being positive. Is that always true?

[Branko_Trifkovic]

No, value s_db_correct is sometimes negative, but that’s because value db (gradient dJ/db) is negative. But I dont know what Im doing wrong

[TMosh]

From the lecture on the Adam optimizer.

Looks like your code is missing the square of the db terms. That’s what lets you take the square root later without fear of imaginary numbers.