In “why do we need activation function” andrew has said that we cant use linear activation everywhere because it will result in just normal linear regression (linear function of a linear function is a linear function) but he also said that we can use Relu activation in every hidden layer. It’s a lit…

Exactly as Tom says, a function is either linear or it’s not and ReLU is “piecewise” linear, but that is nonlinear. It might seem counterintuitive, but it works. You can think of ReLU as the “minimalist” activation function: it’s incredibly cheap to compute and provides just the bare minimum of nonl…

Doubt about Relu activation in hidden layer

Course Q&A Machine Learning Specialization Advanced Learning Algorithms

TMosh January 29, 2023, 4:38pm 2

Relu is not a linear function, because it has a discontinuity at zero, and outputs zero for all negative values.

1 Like

About how to determine the number of layers of the neural network and the number of neurons in each layer

Topic		Replies	Views
Choice of activation function Advanced Learning Algorithms week-2	7	677	November 21, 2022
Activation functions in the hidden layers Advanced Learning Algorithms week-2	4	509	July 21, 2022
Relu activation NLP with Probabilistic Models week-2	1	562	March 14, 2023
Neural Network functions Advanced Learning Algorithms week-2	3	476	April 22, 2023
Why do you need Non-Linear Activation Functions? Neural Networks and Deep Learning	3	683	March 15, 2022

Doubt about Relu activation in hidden layer

Related topics