# Intuition behind RMSprop, GD with moment and Adam **URL:** \ **Category:** Improving Deep Neural Networks: Hyperparameter tun\ **Tags:** coursera-platform\ **Created:** [August 11, 2022, 11:30am UTC](https://community.deeplearning.ai/t/intuition-behind-rmsprop-gd-with-moment-and-adam/182824 "2022-08-11T11:30:55Z")\ **Posts on this page:** 1\ **Showing post:** 3
**Author:** ![paulinpaloalto](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/paulinpaloalto/32/187_2.png) [@paulinpaloalto](https://community.deeplearning.ai/u/paulinpaloalto)\ **Post date:** [August 11, 2022, 11:06pm UTC](https://community.deeplearning.ai/t/intuition-behind-rmsprop-gd-with-moment-and-adam/182824/3 "2022-08-11T23:06:25Z")
> [@David00](#): > > - Why the gradient descent is represented using ellipses? Is the major axis for ‘W’ and the minor axis for ‘b’? There can be tens or hundreds or thousands of features, these weights and biases may not form a simple ellipse. What is the significance of ellipse? In addition to Rashmi’s excellent and detailed response, the point about the pictures is that (exactly as you say) everything here really involves hundreds, thousands or even millions or billions of dimensions. Unfortunately, you can’t draw pictures in more than 3 dimensions and our poor human brains (or at least my poor human brain anyway 🤓) are only capable of visualizing things in 3 dimensions. So the pictures are a very very limited attempt to give some intuition using the extreme limitations of visualizing things in 3D. In addition to all the great links that Rashmi included, [here’s a thread](https://community.deeplearning.ai/t/cost-function-stuck-at-local-minima/95214/4) that has some more discussion about local minima and includes a link to a paper from Yann LeCun’s group about the complexity of loss surfaces. --- _[View the full topic](https://community.deeplearning.ai/t/intuition-behind-rmsprop-gd-with-moment-and-adam/182824)._