Course2 Week3 - Confusion about the example given for linear scale

Course Q&A Deep Learning Specialization Improving Deep Neural Networks: Hyperparameter tun
1

In second video of Week 3, Andrew NG talks about which scales to be used for hyperparameters. He gives an example for linear scale, where the range is between 0.0001 and 1. He says that if we were to use a linear scale %10 of our resources would be go to range between 0.0001 - 0.1 and %90 would be gone to 0.1 - 1.

I didn’t understand what does he mean by resources and how he decided that only %10 is in range 0.0001 - 0.1 when we use a linear scale. I would be happy if someone also give some example about this while explaining it.

1 Like
2

Hi, @mavibirdesmi.

You can think of resources as time. 10% is just the probability of getting a value between 0 and 0.1 if you sample uniformly between 0 and 1.

Check out this post if it’s not clear why you don’t want to sample uniformly.

Good luck with the course :slight_smile:

3

I understood the 10% probability in a linear scale. But can you please explain how by changing to logarithmic scale raises the probability of getting a value within the range 0.0001 and 0.001

4

Hi, @amitp.

Try to follow this example.

In the logarithmic scale case, you’re sampling the exponents uniformly from [-4, 0], so this is the number of points that fall between 0.0001 and 0.1:

image

And this is the number of points that fall in that same interval when you sample uniformly from [0.0001, 1]:

image
image693×126 24.6 KB

Let me know if it’s clear now :slight_smile:

5

Thank you very much for your explanation.

1 Like