Hi, everybody! I need your help to clarify the underlying explanations of questions 4, 6, 7.
Hi @mahmoud_ahc,
Your question is kinda broad, so I’m going to answer 4 and 6 (regarding regularization) broadly. I suggest you re-watch the lecture on Dropout for 7.
Recall that when training a DNN, you could find yourselve in a high-bias|just right|high-variance scenario. Think about how would you spot high-bias (eg. underfitting) and high-variance (overfitting) in your training set and then dev set. Now, the lambda parameter for the regularization has the effect of pushing the weights in a way that could help you alleviate these conditions (eg. move from overfitting to underfitting or viceversa).
Check out the lecture “Why Regularization Reduces Overfitting?” which focuses on this and has an example!
Hope that helps!
2 Likes
@Neurogeek has done a great job covering 4 and 6. For 7, you have to read the question really carefully. The key point is “at test time” and people sometimes forget to focus on that. As Prof Ng discusses in the lectures, any form of regularization only happens during training. At test time you are just doing forward propagation with the generated model to compute the predictions. In fact when you compute the actual training set predictions for measuring training accuracy, you also would not apply regularization (L1, L2, dropout or any other form).