Convolutional Neural Network, Activation functions

Hi, @Moneet_Mohan_Devadig!

Activation functions are applied to the layer’s output. In this case, the activation function is calculated from the output of the first layer and before the second one.

With respect to the second question, activation functions (and precisely non-linear activation functions) are one of the most important parts of a neural network. Without them, the network could only apply linear transformations to the input (very simple), so they give it the ability to approximate non-linear functions, which can be really powerful and perform really well.

1 Like