Week 2, Exercise 4, Optional Assignment (Sigmoid derivative)

Course Q&A Deep Learning Specialization Neural Networks and Deep Learning
1

Hi, the exercise is about calculating the sigmoid derivative. I have implemented the function as:

def sigmoid_derivative(x):
s = 1/(np.exp(-x))

ds = s*(1-s)

return ds

I think am making the mistake while calculating “ds”. I am getting 2 tests passed and 1 test failed message when I run the code.

Any suggestions what could be the issue here?

Thanks

2

Your implementation of the derivative is correct, but the s value that you feed into it is wrong. Why reimplement sigmoid manually here? You already built the function to compute that earlier, so why not save yourself some work by just calling it? But if you insist on building it again, at least compare your current implementation to the previous one. I hope the previous one is different. :grin:

Your implementation of s is:

s = \displaystyle \frac {1}{e^{-x}}

But note that is equivalent to:

s = e^x

right? That’s a little different than sigmoid, no?

1 Like
3

Also,

One quick way to debug these is to find the values of the different variables for different values. We know that the value at infinity (or a very high value) is 1 and the derivative at infinity is 0.

By checking the function, you can see that these are not followed.

1 Like
4

definitely calling the sigmoid function (already implemented) is the better approach. Thanks for the suggestion.

5

Hi,

In this exercise4, I am confused of what we are calculating, why we are calculating the gradient of sigmoid? and why the derivative is equal to
image

is this dg(x)/dx? Why not calculating dj/dw? Or dj/db? Since the dg(x)/dx is not useful

6

I guess this YouTube playlist from @eddy will answer all your questions.

7

Sure, it is our goal to compute \frac {\partial J}{\partial w} and \frac {\partial J}{\partial b}, so the question is how you do that. It involves using the Chain Rule and if you work that out (as Eddy shows on the link that Saif gave us) it involves the derivative of sigmoid as one of the factors.

1 Like