If you use the zoomed-in version of the contour plot with the gradient descent path, then you can see the path is perpendicular to the contour lines. Not sure why the first plot (not zoomed in) doesn’t show the path being perpendicular to the contour lines.
I added screenshots in this post: Should gradient descent create a path that is always perpendicular to contour lines?