HI Sir,
Is the softmax activation functtion dedicated only to the output or last layer ? or can we apply softmax activation function to the hidden layers as well ?
@paulinpaloalto @bahadir @eruzanski @Carina @neurogeek @lucapug @javier @kampamocha
No, softmax is only used at the output layer. With the normal activation functions (sigmoid, tanh, ReLU …), the values of the outputs of each individual neuron are handled separately and there is no connection between them. The point of softmax is that it treats all the inputs as a group and the result is about the relationship between the various values.
If you google this question, here’s a StackExchange article that is the top hit.