Schedule learning rate decay

Course Q&A Deep Learning Specialization Improving Deep Neural Networks: Hyperparameter tun
1

Hey everyone,

(approx. 2 lines)

# if ....:
#     learning_rate = ...
# YOUR CODE STARTS HERE


# YOUR CODE ENDS HERE

Could someone please help me with this part. I tried so hard to understand how to use the if part but I couldn’t figure it out.

Thanks in advance for your help.
Cheers,
Younes

2

Hi, @Younes_hb.

schedule_lr_decay gets called every epoch. But you want to update learning_rate only if the epoch number is a multiple of time_interval.

Did that help? :slight_smile:

1 Like
3

I am having the same problem:
I am using
if isinstance(epoch_num/time_interval, int):
but it doesn’t work.

4

Hi, @fabian.

I guess you were thinking about the mathematical definition of multiple. The problem is:

>>> type(1.0)
<class 'float'>

Think what epoch_num/time_interval being an integer implies. It has nothing to do with Python data types (hint).

5

Of course.
Thank you.

1 Like
6

Hello -

Thanks for the help @nramon, I got the if statement to work but the learning rate isn’t decreasing after the 10th and 100th epochs.

Any thoughts on what I might be missing? Been stuck for awhile. Thanks!

7

Check the condition of the if statement. You need to change the learning rate only when epoch number is a multiple of time interval. Think how can you code this condition.
Hint: modulo operator.

2 Likes
8

The problem was related to this. Enjoy the rest of the course, @cpedrosa!

P.S.: Good hint, @AnkitSaini :slight_smile: