Hi,

Could someone explain to me why the partial derivative of linear regression is the same as that of logistic regression?

The cost function is different for logistic regression.

Regards,

Hi,

Could someone explain to me why the partial derivative of linear regression is the same as that of logistic regression?

The cost function is different for logistic regression.

Regards,

2 Likes

Hello, did you do the derivative yourself to verify that they indeed are the same?

The answer is found in the appropriate application of calculus.

Hi @rmwkwok,

I have tried but without much success. I guess I’ve made a mistake somewhere along the way.

1 Like

Hello @Thrasso00,

Thanks for sharing your work. We can differentiate the following 3 equations separately

l = -y\log{p} - (1-y)\log{(1-p)}

p = \frac{1}{1+\exp{(-z)}}

z = wx

because \frac{\partial{l}}{\partial{w}} = \frac{\partial{l}}{\partial{p}} \frac{\partial{p}}{\partial{z}} \frac{\partial{z}}{\partial{w}}

This should make things easier.

Raymond

3 Likes

Hi @rmwkwok thank you for helping.

Indeed, this is how I have proceeded to arrive at this result. However, I have not been able to prove that the partial derivative is the same. Is there no demonstration of how to arrive at the same result? Surely I must have made a mistake somewhere.

Regards,

Hi @Thrasso00, to differentiate the 3 equations,

\frac{\partial{l}}{\partial{p}} = -\frac{y}{p} + \frac{1-y}{1-p}

\frac{\partial{p}}{\partial{z}} = p(1-p)

\frac{\partial{z}}{\partial{w}} = x

Because \frac{\partial{l}}{\partial{w}} = \frac{\partial{l}}{\partial{p}} \frac{\partial{p}}{\partial{z}} \frac{\partial{z}}{\partial{w}}, we can multiply the above results to get the answer.

Note that p=f

1 Like

Hello @Thrasso00,

Our \log here is e-based, so it would have been better to use \ln instead and your \log{(10)} term can be dropped.

For your second outcome, it is very close to my answer, the trick is to add 1 and minus 1 in the numerator, so that you get a 1 + e^{-z} to cancel with the denominator to make this part p, and the remaining part -p^2.

Cheers,

Raymond

1 Like

Hi @rmwkwok,

Thank you for the information. Then the cost function explained in the course is not correct. It should be something like this:

I’m very sorry but I didn’t quite understand how you simplified the function:

to obtain:

Could you describe the simplification please?

Regards,

1 Like

I forgot where it is but I think it should have been mentioned somewhere that in this course `log`

refers to the natural log.

Change the numerator from e^{-z} into (1+e^{-z})-1, and now the first part has the same form as the denominator. It’s a pretty common tricks. Sometimes I think it is pretty evil. Haha.

Cheers,

Raymond

Thank you very much for your help @rmwkwok now it is clearer.

I imagine it is not necessary to understand these concepts to complete the course but I prefer to understand them.

You are welcome @Thrasso00. I had done the same check myself when I learnt logistic regression.

Raymond