Hello,
I can’t seem to find out how to calculate grad W2 for the assignment. Since it’s not covered clearly in the lecture or in the lecture notes, could someone please post the formula or the code here for the rest of us to understand. It’s really frustrating when you are asked to do the assignment but only given half the info to do it. There isn’t even any hints or helpful comments for this section.
Thank you.
I think you have to use the batch version of the partial derivatives, which are only shown in the video, not the readings. It’s around 2:50 on the video Training a CBOW Model: Backpropagation and Gradient Descent. If you’ve been using the single vector version (not the batch), you’re probably missing the 1/m where m is the batch size.
1 Like
Take a look in the lab notebook C2_W4_lecture_nb_3_training_the_CBOW
See for example the sections around
and later
This really helped me! Previously, I was getting outputs for grad_b1 and grad_b2 that were the incorrect shape, and I was also getting incorrect values for W1 and W2.