Deep Learning Specialization
Course 2: Improving Deep Neural Networks,..
Week 3: Hyperparameter Tuning,…
Quiz, Question 4: Which of the following are true about batch normalization?
One of the correct answers is: “The parameters gamma[l] and beta[l] set the variance and mean of z[l]”. As per my understanding, (I hope I am wrong), gamma is not the variance but the standard deviation. Consequently, shouldn’t this answer be labeled as “Incorrect”?
Regards, Carlos
the statement mentions gamma and beta set the variance, it doesn’t state that gamma is the variance.
I hope you understand the difference between the two sentences, or am I not understanding your query properly?
After the activation z^{[l]} is normalized to have a mean of 0 and a variance of 1, the batch normalization formula applies a linear transformation
Beta avts as the shift parameter (offset). It sets the new mean of the activations, allowing the model to shift the data away from zero if necessary.
gamma actsl as the scale parameter, setting the new variance (or standard deviation) of the activations, allowing the model to change the spread of the data.
\gamma is initialized as 1s and \beta as 0s to begin with a normalized state, then they are learned via backpropagation.
\gamma ^{[l]} and \beta ^{[l]} have dimensions of (n^{[l]}, 1), where n^{[l]} is the number of hidden units in that layer.
If the confusion is between the gamma being variance, here the learnable parameter when sets to new variance which is standard deviations, will help the model to train.
Deepti,
Thanks for the answer. I appreciate the distinction between ‘setting’ and ‘being’ the variance. I understand that you are placing more emphasis on functional phrasing, and I am on technical precision.
My point is that since “Ztilde = Gamma * Znorm + Beta”, Gamma is explicitly the standard deviation. Using the term ‘variance’ in a technical assessment when the parameter is the ‘standard deviation’ creates unnecessary ambiguity for students looking for mathematical rigor. Moreover, since this is a graded test, this phrasing creates a trap for students.
Just trying to help. Kindest Regards,
Carlos
