Selecting optimal learning rate from Learning rate scheduler

[Prasanna_Hegde]

C4_W2_Lab_3_deep_NN

This is what the text says in the notebook:
The generated graph above shows the values of the range of learning rates that leads to lower losses (i.e. sloping downward) and also which ones cause the training to become unstable (i.e. jagged edges and pointing upwards). In general, you will want to pick a point in a downward slope. That means the network is still learning at that point and is stable. Choosing close to the minimum point of the graph will make the training converge to that loss value quicker, as will be shown in the next cells.

but, in the notebook, the lr is selected as 4e-6 from this graph, which I assume should be somewhere in the green region? please correct me if I am wrong. Why not directly take e-5, which is the lowest point of the curve and it appears smooth.

Also, what do the divisions between 10**x represent there? Could you please clarify? why they are not uniformly spaced?

[bruno_ramos_martins]

Hello,

In this section of the notebook, we have:

# Plot the loss in log scale
plt.semilogx(lrs, history.history["loss"])

Due to the code snippet above, in the graph, the values on the x-axis (logarithmic scale) are not linearly spaced, but rather exponentially spaced. This makes the bars appear uneven, even though the original values (in linear scale) are equally spaced.

To make it clearer, you can consult the Matplotlib documentation:
matplotlib.pyplot.semilogy

And regarding the choice of the value, there is a text block that explains that you can choose another value if you wish:

You will then set the optimizer with a learning rate close to the minimum. It is set to 4e-6 initially but feel free to change it based on your results.

Please let me know if this helped with your issue.
Happy studying!