Derivation from softmax regression to logistic regression

In video Professor Andrew Ng said, The softmax regression algorithm is a generalization of logistic regression.

How can the logistic regression model with sigmoid function be derived from Softmax regression model?

Thanks and best Regards
Liyu

1 Like

Hi @liyu,

Consider we use softmax to predict a binary target, we need 2 neurons in the output layer representing the chance for target = 0 and target = 1. Let’s say their output values are z_1 and z_2 respectively,

softmax(z_1, z_2)_1 = \frac{\exp{(z_1)}}{\exp{(z_1)} + \exp{(z_2)}} = \frac{1}{1 + \exp{(z_2 - z_1)}} = sigmoid(z_1 - z_2)

Cheers,
Raymond

2 Likes

oh, yes! Thanks Raymond!
Liyu

2 Likes