In “why do we need activation function” andrew has said that we cant use linear activation everywhere because it will result in just normal linear regression (linear function of a linear function is a linear function) but he also said that we can use Relu activation in every hidden layer. It’s a lit…

Exactly as Tom says, a function is either linear or it’s not and ReLU is “piecewise” linear, but that is nonlinear. It might seem counterintuitive, but it works. You can think of ReLU as the “minimalist” activation function: it’s incredibly cheap to compute and provides just the bare minimum of nonl…

Doubt about Relu activation in hidden layer

Course Q&A Machine Learning Specialization Advanced Learning Algorithms

Christian_Simonis January 29, 2023, 7:58pm 4

In addition to Tom‘s and Paul‘s excellent answers:
We recently had a thread on a similar topic which you probably find interesting. Feel free to take a look!

Happy learning, @cpp219
and best regards
Christian

1 Like

Topic		Replies	Views
Choice of activation function Advanced Learning Algorithms week-2	7	683	November 21, 2022
Activation functions in the hidden layers Advanced Learning Algorithms week-2	4	510	July 21, 2022
Relu activation NLP with Probabilistic Models week-2	1	563	March 14, 2023
Neural Network functions Advanced Learning Algorithms week-2	3	476	April 22, 2023
Why do you need Non-Linear Activation Functions? Neural Networks and Deep Learning	3	683	March 15, 2022

Doubt about Relu activation in hidden layer

Related topics