Hi, I didn’t quite well get the intuition about how in the case of high dimensional dW and db, the algorithm ensures slower update in the horizontal direction and faster update in the vertical direction. It is understandable with the assumption b as vertical and w as horizontal but what exactly hap…

Hi, @ShreS . Sorry for the late reply :sweat: It will simply decrease the step size for large gradients, and increase it for small ones. I don’t think it makes sense to refer to a horizontal or a vertical direction in a high dimensional parameter space (although you can use dimensionality reduction…

Week 2 RMSprop intuition

Course Q&A Deep Learning Specialization Improving Deep Neural Networks: Hyperparameter tun

nramon September 16, 2021, 3:40pm 4

Hi, @jiri.314159265.

Sorry for the late reply

You may find this post interesting.

Just using the sign of the gradient is not free of problems.

Good luck with the course

Topic		Replies	Views
RMSprop in weight update - what if vertical slopes small and horizontal slopes large? Improving Deep Neural Networks: Hyperparameter tun	2	600	September 19, 2021
Question about RMSprop Improving Deep Neural Networks: Hyperparameter tun	1	275	December 17, 2023
Checking Intuition: RMSprop Normalization vs Speed Improvement (Post: RMSprop lecture) Improving Deep Neural Networks: Hyperparameter tun	1	644	October 10, 2022
RMSProp horizontal direction Improving Deep Neural Networks: Hyperparameter tun	1	488	November 2, 2021
RMS prop in a favorable setting Improving Deep Neural Networks: Hyperparameter tun	11	777	September 11, 2021

Week 2 RMSprop intuition

Related topics