Hello @cajumago, I think your direction is right - lowering alpha can prevent the cost to diverge (to overflow). Since you are testing learning parameter values, I suggest you to print out the cost value, say every 100 iteration. If alpha is too large, then you may see that the cost keeps going up (diverge). When it becomes too large, you will see the overflow warning.
If you are trying to establish a “safe range” for alpha, then my another suggestion is, always normalize all of your features so that they all share very similar scales. You can do this for all ML problems in the future. The reason behind the need for normalization is explained here.
Lastly, good to know that you are testing the learning parameters. 
Cheers,
Raymond