A doubt on derivative

Course Q&A Deep Learning Specialization Improving Deep Neural Networks: Hyperparameter tun
1

Dear Mentor,

Could you please guide me on how to calculate this derivative?

image
image1360×501 42.8 KB

From the lecture, the derivative of regularization term is this

image
image785×309 7.39 KB

From my understanding, the answer should be this

image
image893×313 7.73 KB

Please correct me if there is any mistake.

Thank you

2

Hello,

can you share the video link from where you are having this doubt, so we can directly have a look on the required video

Regards
DP

4

Note that in the regularized cost definition, the norm of W is squared.
That is equivalent to the sum of the squares of the individual weight values.

Then you use the power rule (with exponent = 2) to compute the partial derivatives of each weight value.

Note that the cost is defined with a 2 in the denominator, so that it cancels the 2 in the numerator that comes from the power rule.

5

dl/dw is the partial derivative of the loss function for each of the Xs*. It is the rate of change of the loss function to the change in weight.

So when L2 regularisation is applied, this new definition of dw[l] is still a correct definition of the derivative of your cost function, with respect to your parameters, even after adding the extra regularization term at the end and for this reason L2 regularisation is also called weight decay

here W(l) is not summation but partial loss of cost function in relation to partial loss of weight.

6

Hi Ms Deepti_Prasad,

Time: 8:35 / 9:42

7

Hi Mr Tom Mosher,

I still can’t get the correct answer. Could you please guide me based on this attachment?

Hi Ms Deepti_Prasad,

May i know why here W(l) is not summation? Could you please guide me based on this attachment?

image
image783×854 64 KB

Thank you

8

Jason

your image only explains your doubt

DERIVATIVE
DERIVATIVE960×1280 98.6 KB

SORRY FOR THE BAD HAND WRITTING :joy:

Update : I am not deleting this even if it is wrong, so for people who have assumed like me, understand it is incorrect :grinning:
Regards
DP

9

image
image598×700 38.1 KB

Cheers,
Raymond

1 Like
10

ok Raymond so I was wrong in my interpretation :frowning:

1 Like
11

Unfortunately, this step is wrong. :wink:

image

12

The safest way is to consider an element from the Matrix because it makes it a scalar by scalar differentiation. When it is scalar by scalar, everything we have learnt about differentiation will work.

1 Like
13

Thanks for correcting. actually in the regularisation video Professor Andrew only mentions as "it turns dj/dw is /|w(l)/m and doesn’t not explains how. So the question raised by him is genuine.

14

Mr Raymond, Thank you so much for your guidance :grinning:

15

Ms Deepti_Prasad, Thanks for your reply and handwritten note.

16

Mr Tom Mosher, Thanks for your reply.