Gradient Descent Implementation

Course Q&A Machine Learning Specialization Supervised ML: Regression and Classification
1

In the video for Gradient Descent Implementation Andrew talks (@ 1 min 58 secs) about the gradient descent function but how does the partial derivate of the cost function J(w,b) = (1/m(y*log(fx) + (1-y)*log(1-fx)) is reduced to the partial derivate which is same as during linear regression. How does the cost function of logistic regression which is so different than cost function for linear regression but their partial derivates are same.

Screen Shot 2023-03-17 at 10.20.51 PM
Screen Shot 2023-03-17 at 10.20.51 PM1782Ă—900 186 KB

2

Hello @ronnyfrano

Here is a discussion of the same question that I have answered.

Cheers,
Raymond

3

@rmwkwok - I dont have a good understanding of derivates, I was very good at derivatives in my school days but I have lot touch since. Ona different topic should I focus on learning Math related concepts and come back to this specialization?

My roadmap was to first do the MLS specialization and then the Deep learning specialization. Will I be able to understand these courses w/o a good understanding of the underlying Math?

4

You don’t need to understand calculus to use the Machine Learning tools.

5

Hello @ronnyfrano,

  1. Does mastering all Math related concepts guarantee excellent understanding of ML? Nobody can guarantee that. IMHO, crtitical discussions of your understanding and practicing a lot will contribute more.

  2. Is math unnecessary? No. Many algorithms can be explained in Maths why they should work the way their inventors designed them to. If we know Maths, we know better how the algorithms were considered. If we don’t, then for the time being, we fall back to understanding by intuition or whatever explanation given by the lectures.

  3. Is understanding each ML concept to the deepest possible layer necessary for you? I don’t know because it is about you. To me, it is a good-to-have, but it is not the priority.

  4. Can you learn ML and Maths at the same time, or do you have to do one after another? I don’t know because it is about you. To me, I learn as I need.

There is no one single learning path for everyone. However, there are a few things that are likely to happen no matter we do a Math course first or not:

  1. we do not understanding something
  2. we thought we understand something but we are not aware that we don’t actually understand them fully
  3. there are more than 1 way to understand something but understanding them in 1 way is sufficient to move on for now
  4. we understood something but we don’t know how to use them
  5. we used them but we do not know how to explain them

I think the answers to the above 5 questions are discussions and practicing.

I probably have written more than your questions were supposed to be looking for. However, I always take this kind of questions in the angle of “how I would consider it”, because I don’t actually know you well enough to provide very personal suggestion to you.

To me, Maths is a tool, and it is something I can learn everyday.

Cheers,
Raymond

1 Like
6

Wow, what a well thought response. Your response was very empathetic even thought you dont know me. Thanks for that!!!

7

Hi @ronnyfrano,

In case you are interested, I have posted the derivation steps that show the gradients for a linear regression and a logistic regression do look the same.

Cheers,
Raymond