Hi,
Could someone explain to me why the partial derivative of linear regression is the same as that of logistic regression?
The cost function is different for logistic regression.
Regards,
2 Likes
Hello, did you do the derivative yourself to verify that they indeed are the same?
The answer is found in the appropriate application of calculus.
Hi @rmwkwok,
I have tried but without much success. I guess I’ve made a mistake somewhere along the way.
1 Like
Hello @Thrasso00,
Thanks for sharing your work. We can differentiate the following 3 equations separately
l = -y\log{p} - (1-y)\log{(1-p)}
p = \frac{1}{1+\exp{(-z)}}
z = wx
because \frac{\partial{l}}{\partial{w}} = \frac{\partial{l}}{\partial{p}} \frac{\partial{p}}{\partial{z}} \frac{\partial{z}}{\partial{w}}
This should make things easier.
Raymond
3 Likes
@Thrasso00, please do let me know if you need any help or have any question
Hi @rmwkwok thank you for helping.
Indeed, this is how I have proceeded to arrive at this result. However, I have not been able to prove that the partial derivative is the same. Is there no demonstration of how to arrive at the same result? Surely I must have made a mistake somewhere.
Regards,
Hi @Thrasso00, to differentiate the 3 equations,
\frac{\partial{l}}{\partial{p}} = -\frac{y}{p} + \frac{1-y}{1-p}
\frac{\partial{p}}{\partial{z}} = p(1-p)
\frac{\partial{z}}{\partial{w}} = x
Because \frac{\partial{l}}{\partial{w}} = \frac{\partial{l}}{\partial{p}} \frac{\partial{p}}{\partial{z}} \frac{\partial{z}}{\partial{w}}, we can multiply the above results to get the answer.
Note that p=f
1 Like
Hello @Thrasso00,
Our \log here is e-based, so it would have been better to use \ln instead and your \log{(10)} term can be dropped.
For your second outcome, it is very close to my answer, the trick is to add 1 and minus 1 in the numerator, so that you get a 1 + e^{-z} to cancel with the denominator to make this part p, and the remaining part -p^2.
Cheers,
Raymond
1 Like
Hi @rmwkwok,
Thank you for the information. Then the cost function explained in the course is not correct. It should be something like this:
I’m very sorry but I didn’t quite understand how you simplified the function:
to obtain:
Could you describe the simplification please?
Regards,
1 Like
I forgot where it is but I think it should have been mentioned somewhere that in this course log refers to the natural log.
Change the numerator from e^{-z} into (1+e^{-z})-1, and now the first part has the same form as the denominator. It’s a pretty common tricks. Sometimes I think it is pretty evil. Haha.
Cheers,
Raymond
Thank you very much for your help @rmwkwok now it is clearer.
I imagine it is not necessary to understand these concepts to complete the course but I prefer to understand them.
You are welcome @Thrasso00. I had done the same check myself when I learnt logistic regression. 
Raymond