I’m going to train a model for multi-label image classification on a dataset comprising of over 500 thousand images. To get a good performing model, hyperparameter tuning is important. I wish to know if there are any methods that help tune the hyperparameters while training is going on. For example, the model trains for 1 epoch with the initial hyperparameters that I provide, then using some methodology, it adjusts the hyperparameters itself at the start of the 2nd epoch.
As of now, I came across several methods for hyperparameter tuning with the most recent being Bayesian optimization based tuning but it requires retraining the model. Are there any methods for tuning while training?
This is what can help you in doing that and what comes into my mind now. The TF learning rate scheduler.