Course 2 week 2 question on the equation for Adam

Course Q&A Deep Learning Specialization Improving Deep Neural Networks: Hyperparameter tun
1

Hi,

On the update to W (and B)

Do we add epsilon to sdW_corrected and then take the square root or add epsilon afterwards?

I am thinking add it to the term prior to taking the square root

Thank you

Neil

2

Hi, @NeilA.

The formula from the notebook is probably clearer:

image

Good luck with the assignment :slight_smile:

2 Likes
3

Hi NRamon,

Thank you for the response. I was able to get the program to work after correcting the equation!

Thank you,

Neil

1 Like
4

Hi Nramon,

With reference to the lecture notes, Adam puts epsilon outside of the square root, while RMSProp puts it inside the root. Is there a specific reason for the different location?

Thanks!

5

Hi, @Alienway.

Here’s a similar post. I don’t know if there’s a specific reason, but I hope you still find it interesting.

Good luck with the last week of the course :slight_smile:

1 Like