Why we use relu activation function in the hidden layers comapre to other activation functions like sigmoid,linear activation,tanh etc.,

Hi there, benefits are: there is a reduced risk of vanishing gradients since the gradient in the positive section of the ReLU function is constant. It does not saturate in contrast to sigmoid or tanh you can describe well non-linearity as stated in this thread: Isn't Relu just a lineer regression…

In case you are interested in reading more information, also with respect to evaluation of different activation functions, feel free to take a look at this paper: https://arxiv.org/pdf/2109.14545.pdf Best regards Christian

These threads may be interesting, too: Better Activation functions: (tanh > sigmoid) Better Activation Functions (part 2) Best regards Christian

Activation functions

Course Q&A Deep Learning Specialization Convolutional Neural Networks

Christian_Simonis January 4, 2023, 8:16am 4

These threads may be interesting, too:

Better Activation functions: (tanh > sigmoid)
Better Activation Functions (part 2)

Best regards
Christian

Topic		Replies	Views
ReLU vs Sigmoid function Neural Networks and Deep Learning week-module-1 , coursera-platform	2	93	December 24, 2024
Understanding RELU deeply Neural Networks and Deep Learning coursera-platform	6	989	February 5, 2023
DL and NN course1 Week#3: Understanding Activation functions Neural Networks and Deep Learning week-module-3 , coursera-platform	2	66	March 4, 2025
Need of reLU as activation function AI Discussions ai-discussions	3	42	October 24, 2025
ReLU function as activation function Advanced Learning Algorithms week-module-2	3	427	July 11, 2023

Activation functions

Related topics