Hi, On the update to W (and B) Do we add epsilon to sdW_corrected and then take the square root or add epsilon afterwards? I am thinking add it to the term prior to taking the square root Thank you Neil

Hi, @NeilA . The formula from the notebook is probably clearer: [image] Good luck with the assignment :slight_smile:

Hi NRamon, Thank you for the response. I was able to get the program to work after correcting the equation! Thank you, Neil

Hi, @Alienway . Here ’s a similar post. I don’t know if there’s a specific reason, but I hope you still find it interesting. Good luck with the last week of the course :slight_smile:

Course 2 week 2 question on the equation for Adam

Course Q&A Deep Learning Specialization Improving Deep Neural Networks: Hyperparameter tun

Alienway September 4, 2021, 5:34am 4

Hi Nramon,

With reference to the lecture notes, Adam puts epsilon outside of the square root, while RMSProp puts it inside the root. Is there a specific reason for the different location?

Thanks!

Why do you add epsilon in the denominator of the final updating step of Adam optimization?

Topic		Replies	Views
C2 W2: Improving Deep Neural Networks Week 2 Programming Assignment Improving Deep Neural Networks: Hyperparameter tun coursera-platform	11	592	April 6, 2024
DLS Course 2 week 2 Programming Assignment Optimization methods Exercise 6 update parameters with adam Improving Deep Neural Networks: Hyperparameter tun coursera-platform	4	674	August 11, 2022
Why do you add epsilon in the denominator of the final updating step of Adam optimization? Improving Deep Neural Networks: Hyperparameter tun coursera-platform	3	640	April 23, 2023
Course 2 week 2 update parameters with adam Improving Deep Neural Networks: Hyperparameter tun coursera-platform	3	672	August 12, 2021
Update_parameters_with_adam Improving Deep Neural Networks: Hyperparameter tun coursera-platform	4	722	June 16, 2021

Course 2 week 2 question on the equation for Adam

Related topics