Difference between loss function and cost function?

Course Q&A Machine Learning Specialization Advanced Learning Algorithms
2

Hi there,

often, the terms loss and cost functions are used interchangeably even though they are not 100% synonyms.

More specifically:

  • cost function is more associated with optimization. Sometimes, costs can also refer to evaluating the optimization (resp. the model)
  • loss is usually referring to the training of a model, usually a ML model. Here, of course also an optimization is solved with loss as the criterion (= costs, see previous point. We want to minimize the costs by fitting the model parameters during training), see also this thread: How does a Deep Neural Network work? - #4 by Christian_Simonis

Hope that helps, @Ahmad_Fareed_Khan!

Best regards
Christian

3 Likes