How did he choose lr value at 1:39 in this video “deep neural network”
link to video: https://www.coursera.org/learn/tensorflow-sequences-time-series-and-prediction/lecture/KcQGM/deep-neural-network
I watched that too but I still don’t get it. Can you elaborate a little, please?
Hi @talhairfan,
What we are looking here is an optimal value for the learning rate. One technique would be to test with different values and check which one fits better - i.e. gives a lower loss value.
So we implement a callback function that will be called at the end of each epoch. Only thing it does is to update learning rate. If we measure how good is our fit - loss value - when we change a bit learning rate, you can find minimum value that keeps loss minimum - good fit - in a stable position. Is at that point where we can asume that learning rate is a good value.
Hope it helps.
Best,
Thanks, I get it. I see the we pick the lowest point of the curve where it’s relatively
stable. But how do we calculate the value of the lr that we are going to use next? take this graph for example:
y-axis: loss
x-axis: learning rate
value picked: 7 x 10^(-6)
Hi @talhairfan,
I believe I’ve got your point… forgive me if I’m wrong.
Looking at the graphic, lower value seems to be 3 10E-5. Notice we are reading in logarithm scale on the x-axis and lower values are on the left.
Best,
Thank you so much. I got it.