In video Professor Andrew Ng said, The softmax regression algorithm is a generalization of logistic regression.

How can the logistic regression model with sigmoid function be derived from Softmax regression model?

Thanks and best Regards

Liyu

In video Professor Andrew Ng said, The softmax regression algorithm is a generalization of logistic regression.

How can the logistic regression model with sigmoid function be derived from Softmax regression model?

Thanks and best Regards

Liyu

1 Like

Hi @liyu,

Consider we use softmax to predict a binary target, we need 2 neurons in the output layer representing the chance for target = 0 and target = 1. Let’s say their output values are z_1 and z_2 respectively,

softmax(z_1, z_2)_1 = \frac{\exp{(z_1)}}{\exp{(z_1)} + \exp{(z_2)}} = \frac{1}{1 + \exp{(z_2 - z_1)}} = sigmoid(z_1 - z_2)

Cheers,

Raymond

2 Likes

oh, yes! Thanks Raymond!

Liyu

2 Likes