Kindly clarify activation vector for a given layer of NN.
z- input to the activation function
a- output of the activation function
At every layer of a neural network, you must have an activation function g^{[l]}(z), which is a non-linear function that takes the “linear activation” result z of a given layer and converts that to the actual output of the layer:
a = g(z)
For the hidden layers of a neural network, we have lots of choices for the activation function: ReLU, Leaky ReLU, tanh, sigmoid, swish and more. For the output layer of a neural network that does classification, the activation is not a choice: we use sigmoid for binary (“yes/no”) classifications and softmax for multiclass classifications (where the output is one of a number of “classes”, e.g. cat, dog, cow, horse, pig, squirrel, elephant, kangaroo …).