Why we use relu activation function in the hidden layers comapre to other activation functions like sigmoid,linear activation,tanh etc.,

Hi there, benefits are: there is a reduced risk of vanishing gradients since the gradient in the positive section of the ReLU function is constant. It does not saturate in contrast to sigmoid or tanh you can describe well non-linearity as stated in this thread: Isn't Relu just a lineer regression…

Activation functions

Course Q&A Deep Learning Specialization Convolutional Neural Networks

Christian_Simonis January 4, 2023, 8:16am 4

These threads may be interesting, too:

Better Activation functions: (tanh > sigmoid)
Better Activation Functions (part 2)

Best regards
Christian

Topic		Replies	Views
Choice of activation function Advanced Learning Algorithms week-2	7	667	November 21, 2022
Activation functions in the hidden layers Advanced Learning Algorithms week-2	4	507	July 21, 2022
ReLU function as activation function Advanced Learning Algorithms week-2	3	420	July 11, 2023
Neural Network functions Advanced Learning Algorithms week-2	3	474	April 22, 2023
Activation function in NN NLP with Classification and Vector Spaces week-3	3	327	March 30, 2022

Activation functions

Related topics