When I run my implementation of adam optimization, got this error (s_db_correct is negative value). And I dont know where is a problem

Here is implementation of adam algorithm

And this is function where I compute gradients for w and b

When I run my implementation of adam optimization, got this error (s_db_correct is negative value). And I dont know where is a problem

Here is implementation of adam algorithm

And this is function where I compute gradients for w and b

1 Like

Your code in update() seems to rely on the value of s_db_correct always being positive. Is that always true?

1 Like

No, value s_db_correct is sometimes negative, but that’s because value db (gradient dJ/db) is negative. But I dont know what Im doing wrong

1 Like