# Derivative of "Simplified Cost Function"

**URL:** <https://community.deeplearning.ai/t/derivative-of-simplified-cost-function/292832>\
**Category:** Supervised ML: Regression and Classification\
**Tags:** week-module-3\
**Created:** [March 19, 2023, 8:59am UTC](https://community.deeplearning.ai/t/derivative-of-simplified-cost-function/292832 "2023-03-19T08:59:54Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Arne\_Schirmacher](https://avatars.discourse-cdn.com/v4/letter/a/9fc29f/32.png) [@Arne\_Schirmacher](https://community.deeplearning.ai/u/Arne_Schirmacher)\
**Post date:** [March 19, 2023, 8:59am UTC](https://community.deeplearning.ai/t/derivative-of-simplified-cost-function/292832/1 "2023-03-19T08:59:54Z")

</div>

In the workbook C1\_W3\_Lab06\_Gradient\_Descent\_Soln, the instructor explains that w\_j and b are updated like this:  
w\_j = w\_j - \alpha \frac{\partial J(\mathbf{w},b)}{\partial w\_j}

b = b - \alpha \frac{\partial J(\mathbf{w},b)}{\partial b}

The corresponding code looks pretty much like the calculating gradients for linear regression but with an extra call to the sigmoid function. Note that the original linear regression code assumes that J(w,b) uses the least squares method.

```auto
    for i in range(m):
        f_wb_i = sigmoid(np.dot(X[i],w) + b) #(n,)(n,)=scalar
        err_i = f_wb_i - y[i] #scalar
        for j in range(n):
            dj_dw[j] = dj_dw[j] + err_i * X[i,j] #scalar
        dj_db = dj_db + err_i
    dj_dw = dj_dw/m #(n,)
    dj_db = dj_db/m #scalar

```

However the compute\_cost\_logistic function uses a different cost function which is shown in video " Simplified Cost Function for Logistic Regression". The video does not explain how \frac{\partial{J(w,b)}}{\partial{w}} and \frac{\partial{J(w,b)}}{\partial{b}} look in this case.

Here is the derivative to \frac{\partial{J(w,b)}}{\partial{w}}, it is not equivalent to the code listed above:

> **[d/dw of y\*ln(1/(1+e^-(wx+b)))+(1-y)ln(1-(1/(1+e^-(wx+b)))) - Wolfram|Alpha](https://www.wolframalpha.com/input?i=d%2Fdw%2Bof%2By*ln%281%2F%281%2Be%5E-%28wx%2Bb%29%29%29%2B%281-y%29ln%281-%281%2F%281%2Be%5E-%28wx%2Bb%29%29%29%29)**
>
> Wolfram|Alpha brings expert-level knowledge and capabilities to the broadest possible range of people—spanning all professions and education levels.

Please explain why the example code does not use the actual derivative. What am I missing?

---

<div class="post-metadata">

**Author:** ![rmwkwok](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/rmwkwok/32/126977_2.png) [@rmwkwok](https://community.deeplearning.ai/u/rmwkwok)\
**Post date:** [March 19, 2023, 10:24am UTC](https://community.deeplearning.ai/t/derivative-of-simplified-cost-function/292832/2 "2023-03-19T10:24:11Z")

</div>

Hi @Arne_Schirmacher

Please refer to the following post for the derivation steps that show the gradients for a linear regression and a logistic regression do look the same.

> [@Summary and the derivations of gradients for linear regression and logistic regression](https://community.deeplearning.ai/t/summary-and-derivation-of-gradients-for-linear-regression-and-logistic-regression/292863):
>
> Hello! This post first [tabulates](https://community.deeplearning.ai/t/summary-and-derivation-of-gradients-for-linear-regression-and-logistic-regression/292863#table-1-model-and-gradients-formula-1) the gradients for a linear regression problem and a logistic regression problem. For those who are interested and are familiar with differentiation and chain rule, the derivation steps for the gradients are also shown after the table. For the sake of comparing the derivation steps, I have made the [second table](https://community.deeplearning.ai/t/summary-and-derivation-of-gradients-for-linear-regression-and-logistic-regression/292863#table-2-comparing-the-derivation-steps-4). One important take-away is that the gradients for the linear regression and the logistic regression do look the same which can be proven by the derivat…

In particular, if you look at the second table, you will see clearly that the denominator term generated in the Logistic Regression due to the loss function gets canceled out, thanks to the Sigmoid function. After the cancellation, the terms between linear regression and logistic regression no longer look any different.

 ![image](https://global.discourse-cdn.com/dlai/original/3X/9/1/9102c3ba21c73372463d1d9c920a95dc8bb15bda.png)

Cheers,  
Raymond