It’s a perceptive question! I’m not a mentor for MLS and don’t know what Professor Ng says about convergence there, but this is discussed also in DLS (the next series you probably would take after MLS). There are a couple of high level points worth mentioning:
- There is no such thing as guaranteed convergence once you get to real Neural Networks. The solution surfaces are too complex. Here’s a DLS thread about this.
- Yes, there are more sophisticated techniques for dynamically managing the incremental learning rate based on derivatives and other similar techniques. If you want a preview of some of those ideas, try googling “Adam optimization” and get an idea how that works. Adam is one of the algorithms covered in DLS C2.
So “hold that thought” and take DLS next and you’ll learn more about all that.