During the lesson regarding Logistic Regression Gradient Descent, the teacher describes the derivative of the loss function respective to the inputs. I have tried to learn derivatives myself to gain a better insight into what actually goes into an AI but upon trying to achieve the answer myself I am getting stuck on where I went wrong on the calculations. I learnt most of derivative myself so any insight would be extremely helpful!
I am calculating the derivative of this
-ylog(a)+(1-y)log(1-a)
Here’s a thread that shows how to calculate many of the relevant derivatives for the loss functions and activation functions.
One thing to note here is that the notation in the ML world is different than in the “pure math” world. When they say log here, they mean the natural log, not the log base 10. So there should be no constants of ln(10) required anywhere here.
Hello @michael_yoon
Besides the above, it is better for you to specify with respect to (w.r.t.) what you are taking the derivatives. For example, if you are taking them w.r.t. a, then do not write y', but \frac{\partial{y}}{\partial{a}} instead. This forces us to be aware of what needs to be considered. For example, after you write down \frac{\partial{y}}{\partial{a}}, then you think about what y and a are AND whether y depends on a or not - if so, you need to find out what that dependence is; if not, you know the answer for that derivative should be 0.
Writing things down explicity and clearly is a very powerful way for doing things right. You have written things very clearly 
, but we can do a little bit more on the “explicit” part.
I suggest you to adopt my suggested practice, redo the calculation, and check your new steps with Paul’s link.
Cheers,
Raymond
