what if I have two tasks one is regression and the other is a classification ,how would I formulate the cost function
For a regression task, you probably want a “distance” based cost function like MSE. For classification, you need some form of “cross entropy” loss. Then you could make the overall cost function be either the sum or the average of those two separate costs. If you minimize A + B, it’s the same as minimizing (A + B)/2, meaning that the coefficients of the solution will be the same in both cases, right?