# Why softmax in last layer for multiclass NN? **URL:** \ **Category:** Improving Deep Neural Networks: Hyperparameter tun\ **Tags:** coursera-platform\ **Created:** [January 7, 2022, 1:18am UTC](https://community.deeplearning.ai/t/why-softmax-in-last-layer-for-multiclass-nn/83645 "2022-01-07T01:18:17Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![Vishrut\_Pandey](https://avatars.discourse-cdn.com/v4/letter/v/839c29/32.png) [@Vishrut\_Pandey](https://community.deeplearning.ai/u/Vishrut_Pandey)\ **Post date:** [January 7, 2022, 1:18am UTC](https://community.deeplearning.ai/t/why-softmax-in-last-layer-for-multiclass-nn/83645/1 "2022-01-07T01:18:18Z")
Why do we use softmax at last layer? Our prediction will simply be the class whose corresponding Z value was maximum(before softmax is applied). One reason I think of is help with back prop since softmax will have a well defined derivative. Is there anything else I am missing out? ---
**Author:** ![Deepika\_Jindal](https://avatars.discourse-cdn.com/v4/letter/d/71c47a/32.png) [@Deepika\_Jindal](https://community.deeplearning.ai/u/Deepika_Jindal)\ **Post date:** [January 7, 2022, 2:18am UTC](https://community.deeplearning.ai/t/why-softmax-in-last-layer-for-multiclass-nn/83645/2 "2022-01-07T02:18:37Z")
The softmax function is used as the activation function in the output layer of neural network models that predict a multinomial probability distribution. That is, softmax is used as the activation function for multi-class classification problems where class membership is required on more than two class labels. ---
**Author:** ![Vishrut\_Pandey](https://avatars.discourse-cdn.com/v4/letter/v/839c29/32.png) [@Vishrut\_Pandey](https://community.deeplearning.ai/u/Vishrut_Pandey)\ **Post date:** [January 7, 2022, 4:00am UTC](https://community.deeplearning.ai/t/why-softmax-in-last-layer-for-multiclass-nn/83645/3 "2022-01-07T04:00:05Z")
Thanks for the answer. But lets say when we have to predict the output class in a multiclass classification, why can’t we simply see the highest Z value(like 5 in [5,4,1,2,3]) and predict our our put with that. Softmax will anyway be predicting the class with maximum Z value so why do the extra step of applying softmax? ---
**Author:** ![paulinpaloalto](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/paulinpaloalto/32/187_2.png) [@paulinpaloalto](https://community.deeplearning.ai/u/paulinpaloalto)\ **Post date:** [January 7, 2022, 4:32am UTC](https://community.deeplearning.ai/t/why-softmax-in-last-layer-for-multiclass-nn/83645/4 "2022-01-07T04:32:54Z")
You can think of _softmax_ as the multi-class generalization of _sigmoid_. You’re right that you could simply select the highest Z value as the prediction, but then how do you compute a loss function with that method? If you use _softmax_ which “normalizes” the prediction outputs to look like a probability distribution, you can then use the cross entropy loss function and the mathematical properties are identical to what you get in the binary classification case with _sigmoid_ and cross entropy loss. ---
**Author:** ![paulinpaloalto](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/paulinpaloalto/32/187_2.png) [@paulinpaloalto](https://community.deeplearning.ai/u/paulinpaloalto)\ **Post date:** [January 7, 2022, 4:34am UTC](https://community.deeplearning.ai/t/why-softmax-in-last-layer-for-multiclass-nn/83645/5 "2022-01-07T04:34:42Z")
In addition to the material that Prof Ng provides here about softmax, you might also find it useful to watch Prof Geoff Hinton’s lecture about softmax and cross entropy loss. You can find it [on YouTube here.](https://www.youtube.com/watch?v=mlaLLQofmR8) ---
**Author:** ![Vishrut\_Pandey](https://avatars.discourse-cdn.com/v4/letter/v/839c29/32.png) [@Vishrut\_Pandey](https://community.deeplearning.ai/u/Vishrut_Pandey)\ **Post date:** [January 7, 2022, 5:34am UTC](https://community.deeplearning.ai/t/why-softmax-in-last-layer-for-multiclass-nn/83645/6 "2022-01-07T05:34:51Z")
Thanks a lot for the detailed answer.