Adaptive Learning Rate Decay

Course Q&A Deep Learning Specialization Improving Deep Neural Networks: Hyperparameter tun
1

Dear Mentor,

Could you suggest some literature on adaptive learning rate decay methods, as mentioned in the programming assignment of Week 2, Course 2.

Thanks

2

Hi Sankha,

Welcome to the community!

Here’s a few links that could be of use. Well, you can always dig more through web search.

How Does Learning Rate Decay Help Modern Neural Networks?

Adaptive Learning Rate Method - Convolutional Neural Networks for Image and Video Processing - TUM WikiAdaptive Learning Rate Method

AN EMPIRICAL STUDY OF LEARNING RATES IN DEEP NEURAL NETWORKS FOR
SPEECH RECOGNITION

Keep Learning.