About tuning learning rate in c4w4 assignment

[anon76241992]

hello there

I just finished the c4w4 assignment and I have a question about this text.

" Notice that this is only changing the learning rate during the training process to give you an idea of what a reasonable learning rate is and should not be confused with selecting the best learning rate, this is known as hyperparameter optimization and it is outside the scope of this course."

So I do not understand why picking an optimal learning rate by running experiment training one time to see how learning change is not a way of selecting the best learning rate. Why is it different from hyper-tuning?? or does the process of hyperparameter optimization needs to be done via grid-search or hyperopt method only???

[gent.spah]

It is a hyperparameter search nonetheless but it is blind and it might lead you to a better direction or it might. The other automated type of searches like grid or random are more structured and can yield better results with fewer resources utilized, that why he is suggesting to use a systemic approach about it. These methods include several hyper parameter selections and can be compared with one other much easier.