Week 2 Quiz - issue with question

Course Q&A Deep Learning Specialization Improving Deep Neural Networks: Hyperparameter tun
1

Available answers to Question 5 of my quiz do not make sense. None of them seem correct according to the formulas from the lecture. What does prime sign mean?

Question 5 Issue - Answer
Question 5 Issue - Answer1139×885 37.4 KB

Sorry for posting the question, but answer is wrong anyway. Please delete it if not allowed.

2

As we learned, “exponentially weighted averages” is useful to see a trend from noisy data. As you see, given equations are;

v_0 = 0, \ \ \ v_t = \beta v_{t-1} + (1-\beta)\theta_t, \ \ \ \beta = 0.5

Let’s think about the case just using the above equation.

v_0 = 0 \\ v_1 = \beta v_0 + (1-\beta)\theta_1 = 0.5*0 + 0.5*10 = .... \\ v_2 = \beta v_1 + (1-\beta)\theta_2 = 0.5*v_1 + 0.5*10 = ....

Using “weights”, we calculate “temperature” for a trend analysis from yesterday and today’s temperature. I think you can get the answer from the above easily.

As we started with v_0 = 0, this trend curve started from a lower value since it is an exponentially weighted average of \theta_1 and 0. We need to think about a bias term.
Andrew proposed a following equation for Bias correction, \frac{v_t}{1-\beta^t} instead of vt.
So, the above equations are now;

v_0 = v_0^{corrected} = 0, \ \ \ v_t^{corrected} =\frac{ \beta v_{t-1}^{correctetd} + (1-\beta)\theta_t}{1-\beta^t}, \ \ \ \beta = 0.5

Then, you can calculate “temperature” just like we did without bias correction.

So, there is the answer.

But, I realized that confusions may come from a “prime sign” that you pointed out. This is used in different ways in math, like “transpose”, “derivative”, .
In this case, I suppose this is either typo or problem in interpreting LaTex text in a browser. This must be “comma” to separate v_2 and v_2^{corrected}.

Sorry about this misleading prime sign…

3

OK, this clarifies it. The lecture was not very clear about it. My understanding from the lecture was that you calculate v1, v2, … using regular formula, then apply corrections. Instead, you apply correction for each step, then use it as input in the next one.
It seems that in the offered answers, instead of 7.5’, it should be 0.75.

4

Just for clarification about a prime sign issue.

v_2 = 7.5' \ v2^{corrected}=7.5 \ \ \ should be,
v_2 = 7.5, \ v2^{corrected}=7.5 \ \ \

6

From what my version of this quiz, it seems they are using the v_(t-1) instead of v_(t-1)^corrected for calculation. And in the coding assignment for ADAM, v_(t-1) is also used instead of v_(t-1)^corrected. So I guess in the lecture when Andrew first introduced bias correction, he can make it clear that the correction is done afterwards and you still get vt using the uncorrected v_(t-1)