How to come up intuition about selecting a function as loss/ regression

So throughout the course the concepts are quite easy to grasp, but what I was wondering was how to come with such concepts?
For example in the cost function for logistical regression we use -log() . Why we use it is understood, but how can we come up with something like that?

If math is needed, then to which level?
Because I feel that the lack of intuition of coming up with such details might hinder future growth in advanced areas

You don’t need to invent your own cost functions.

Each method (such as linear regression or logistic regression) has its own cost function which has lots of experience and is known to work well.

1 Like

And if you want to invent your own cost function (as many engineering problems need a custom cost function to implement domain knowledge and honor physics laws), then you need the knowledge of Partial Differential Equations (PDE). Prof. Maziar Raissi introduced a Physics Informed Neural Network (PINN) in 2017. You can read his research papers on this topic.

Thanks for the reply!
I wanted to know the intuition behind coming up with such algorithms (not just cost functions)

An analogy would be that we can use sklearn to made models in one line, but it is better to learn from scratch in numpy to understand the learning.

Maybe reading this book (Chapter 4) and this research paper will answer your question.

Thanks a lot! Will definitely go thru the resources.
Sorry but there seems to be a problem with the book’s link
Can you please name the book?

I updated my previous reply and now the link should work. Let me know if it didn’t…