Loss function for Multiclass classification for Logistics Regression

When creating a post, please add:

loss(f_{\mathbf{w},b}(\mathbf{x}^{(i)}), y^{(i)}) =
- log(f_{\mathbf{w},b}( \mathbf{x}^{(i)} )) & {if y^{(i)}=1}
- log ( 1 - f_{\mathbf{w},b}( \mathbf{x}^{(i)} ) ) & {if y^{(i)}=0}

  • f_{\mathbf{w},b}(\mathbf{x}^{(i)}) is the model’s prediction, while y^{(i)} is the target value.

  • f_{\mathbf{w},b}(\mathbf{x}^{(i)}) = g(\mathbf{w} \cdot\mathbf{x}^{(i)}+b) where function g is the sigmoid function.

The loss function above can be rewritten to be easier to implement.
loss(f_{\mathbf{w},b}(\mathbf{x}^{(i)}), y^{(i)}) = (-y^{(i)} log(f_{\mathbf{w},b}( \mathbf{x}^{(i)} ) ) - ( 1 - y^{(i)}) log ( 1 - f_{\mathbf{w},b}( \mathbf{x}^{(i)} ) )

This is a rather formidable-looking equation. It is less daunting when you consider y^{(i)} can have only two values, 0 and 1. One can then consider the equation in two pieces:
when y^{(i)} = 0, the left-hand term is eliminated:

\begin{align} loss(f_{\mathbf{w},b}(\mathbf{x}^{(i)}), 0) &= (-(0) \log\left(f_{\mathbf{w},b}\left( \mathbf{x}^{(i)} \right) \right) - \left( 1 - 0\right) \log \left( 1 - f_{\mathbf{w},b}\left( \mathbf{x}^{(i)} \right) \right) \\ &= -\log \left( 1 - f_{\mathbf{w},b}\left( \mathbf{x}^{(i)} \right) \right) \end{align}

and when y^{(i)} = 1, the right-hand term is eliminated:

\begin{align} loss(f_{\mathbf{w},b}(\mathbf{x}^{(i)}), 1) &= (-(1) \log\left(f_{\mathbf{w},b}\left( \mathbf{x}^{(i)} \right) \right) - \left( 1 - 1\right) \log \left( 1 - f_{\mathbf{w},b}\left( \mathbf{x}^{(i)} \right) \right)\\ &= -\log\left(f_{\mathbf{w},b}\left( \mathbf{x}^{(i)} \right) \right) \end{align}

My question here how to implement or calculate Loss function and cost function for Multiclass classification using Logistic Regression. Please help me here in the same thread.

It’s covered later in the course.

Essentially, you create multiple outputs, and each one identifies one of the labels.

The cost is the total of all of the separate outputs.

Will it be cover in the same course or same specialization under different course?

MLS, Course 2, Week 2.

Thank you so much. :slight_smile: