How we decide the learning rate and number of iterations.

I know what they are, and what they do. but I don’t know how can I determine their values depending on the problem

Hello @Mohamed_Mostafa3

Welcome to the community.

The fact is that it involves a trial and error process to determine the value of alpha and number of iterations - And this can vary from dataset to dataset.

In the case of number of iterations, if we see that there is significant change in cost still happening when we exit the loop, then we should increase the number of iterations and re-run it.

In case of learning rate, we would like it to be a relatively low value so that the update of w does not bounce around. However, we dont want it to be so low that the update of w moves at ridiculously slow pace that it could take forever to meet the convergence condition.