How the visualization convey the right picture?

I think the example of Mount Fuji taken during the comparison of minimizing J(w) in the case of house price regression gives a wrong picture. J(w) curves inside and minimizing J(w) amounts to moving towards the center of the concentric circles, whereas the Mount increases in height as you go towards the center… I’m unable to see how this is the correct comparison.

Please, correct me if I’m wrong.

Hello @Anand_Iyer, welcome to our community. Can you also share the name of the video that you are talking about?

Raymond

Thanks for the reply @rmwkwok

I’m talking about this video - https://www.coursera.org/learn/machine-learning/lecture/QI1h6/visualizing-the-cost-function (4:30), where Prof. speaks of a hammock and goes on to mention Mount Fuji in his next slide. While the hammock represents a valley and hence gradient descent is apt , how does that work on a peak like Mount Fuji?

I see. Thank you @Anand_Iyer for sharing the link and the timestamp.

I think the analogy is not about they are both valley. Instead it’s about the concept of contour lines and that they both have a “center” except that it is a lowest point for cost and it is a highest point for the example of Mount Fuji.

What’s in common between the bowl shape cost and the Mount Fuji is

  • the contour line is (for Mount Fuji) and can be (for linear regression cost) circular around a “center”
  • there is gradient between 2 contour lines (decreasing cost towards the “center” for Cost, and increasing height towards the “center” for Mount Fuji)
  • can use the gradient as a guide to the “center” (gradient descent for Cost, gradient asscent for Mount Fuji)

I think the above is what we can see from this analogy.

Cheers,
Raymond

2 Likes