Hi,
So I wanted to ask about the part where the fraction error in binary classification is computed using the ‘misclassified/num_predictions’ fraction, can we use the loss function there instead (with the log) , just like we used the mean squared error for the regression problem ?
Thanks
Hello @yasmine3,
The short answer is Yes!
We give the name “metric function” to the method for model evaluation. However, we can basically use any function, including the loss function itself, for the metric function as long as we think it is suitable.
The lab used misclassified rate as the metric function probably because it’s easier to understand. We can imagine that reporting it to a business manager is more explainable than reporting the loss value, isn’t it? For example, we can speak to the manager that we have come up with a model that done very well which only misclassified 2% of samples.
However, using the loss, as you have asked, is completely fine too.
Cheers,
Raymond
oh alright that is clear.
And in practice which one is better? Is the loss function considered faster and more robust for example? because in real life problems ( i mean more professional), I find them using the loss method more
Hello @yasmine3,
In real life, I think we should use the metric set by our project goal. If the goal is to achieve an error rate of 1%, then I will use the error rate.
Then we may ask, why don’t we use the error rate for both loss function and metric function? Because we can choose any function as the metric function but not any function as the loss function!
A valid loss function needs to be “differentiable”. “Differentiable” means we can compute the gradients. We need to compute the gradients because we have learnt that we need the gradients to do gradient descent. Error rate is not differentiable, so we cannot use it as loss function.
If we are training a model that serve for a specific project goal, it is the best choice to use the metric that can measure the project goal as metric function, isn’t it? If my client wants the model to do well on the error rate or some very strange metric, then that is the metric function. The client can make strange metric but that metric doesn’t necessarily suitable as a loss function when it is not differentiable.
If we are training a model with no specific goal (no specific metric), then we may just use the loss as metric. In the context of this discussion, a goal means a quantifiable goal which means a metric function.
Cheers,
Raymond
Thank you very much, that was clear
You are welcome, @yasmine3