For positive values, the ReLU function is identical to Linear Function. Then, in cases where our inputs are all positive features, how can a Neural Network provide non-linear decision boundaries?

Some of your weights are going to be negatives, though it depends on how you initialize your weights. Even if all your initial weights and features are positive, after some rounds of gradient descents, some weights might still become negative. Once some weights are negative, it is possible to trigge…

Why do we need an activation function? | ReLU activation

Course Q&A Machine Learning Specialization Advanced Learning Algorithms

alvaroramajo August 2, 2022, 9:08am 5

Hi, @ankitprashnani!

Check this post about activation functions.

1 Like

Topic		Replies	Views
Why do we need Activation function Neural Networks and Deep Learning coursera-platform	4	544	February 16, 2023
C1_W3-Non-Linear_Activation_Function Neural Networks and Deep Learning coursera-platform	1	550	May 18, 2021
Differences between ReLU and linear for positive values Advanced Learning Algorithms week-module-2	4	731	January 16, 2023
Why is ReLU any better than Linear Neural Networks and Deep Learning coursera-platform	7	878	May 2, 2021
Relu activation NLP with Probabilistic Models week-module-2	1	565	March 14, 2023

Why do we need an activation function? | ReLU activation

Related topics