Derivative of "Simplified Cost Function"

[rmwkwok]

Hi @Arne_Schirmacher

Please refer to the following post for the derivation steps that show the gradients for a linear regression and a logistic regression do look the same.

Summary and the derivations of gradients for linear regression and logistic regression

Hello! This post first tabulates the gradients for a linear regression problem and a logistic regression problem. For those who are interested and are familiar with differentiation and chain rule, the derivation steps for the gradients are also shown after the table. For the sake of comparing the derivation steps, I have made the second table. One important take-away is that the gradients for the linear regression and the logistic regression do look the same which can be proven by the derivat…

In particular, if you look at the second table, you will see clearly that the denominator term generated in the Logistic Regression due to the loss function gets canceled out, thanks to the Sigmoid function. After the cancellation, the terms between linear regression and logistic regression no longer look any different.

Cheers,
Raymond