How is the derivative s*(1-s) in sigmoid_derivative

Course Q&A Deep Learning Specialization Neural Networks and Deep Learning
1

In the week 2 assignment titled β€œpython basics with numpy”, when defining sigmoid_derivative function, the derivate is defined with below formula
π‘ π‘–π‘”π‘šπ‘œπ‘–π‘‘_π‘‘π‘’π‘Ÿπ‘–π‘£π‘Žπ‘‘π‘–π‘£π‘’(π‘₯)=πœŽβ€²(π‘₯)=𝜎(π‘₯)(1βˆ’πœŽ(π‘₯))

I am unable to understand:

  1. This derivate is measuring rate of change of β€œx” w.r.t what? Derivate
  2. How is the this formula derived.
2

\sigma(x) = \frac{1}{1+e^{-x}}

\sigma^\prime(x) = \frac{d}{dx}\sigma(x) = \sigma(x)(1-\sigma(x))

Here’s an image of the derivation I found on the web. Haven’t checked it carefully, but it starts and ends at the right place, so should give you a decent idea of what’s going on…

image
image528Γ—569 33.3 KB

3

The derivative of the function \sigma(x) gives the rate of change of the output of the function w.r.t. the input variable x. Ai_curious has given you the derivation. Just googling β€œsigmoid derivative” will get you plenty more.

1 Like