How does RELU activation achieve nonlinearity, or to say how can Relu activation in neural networks fit non linear data, or be better than the other activation functions, because it is eventually linear with only a rectified output ? Why not a 2nd degree polynomial activation function ? or even a si…

Understanding RELU deeply

Course Q&A Deep Learning Specialization Neural Networks and Deep Learning

Christian_Simonis February 4, 2023, 8:07pm 6

In addition to Paul’s excellent explanation: if you are searching for more info and additional sources: here some thread which could be interesting for you, @Ashwin_Yellore:

Happy learning and all the best
Christian

Topic		Replies	Views
Choice of activation function Advanced Learning Algorithms week-module-2	7	693	November 21, 2022
Relu activation NLP with Probabilistic Models week-module-2	1	567	March 14, 2023
How non linear is ReLU? Neural Networks and Deep Learning coursera-platform	4	808	March 17, 2023
Doubt about Relu activation in hidden layer Advanced Learning Algorithms week-module-2	3	840	January 29, 2023
Activation functions Convolutional Neural Networks coursera-platform	3	738	January 4, 2023

Understanding RELU deeply

Related topics