From my understanding, “relu” function is same with linear regression except y = 0 with x <= 0. So it will easily lead to high bias instead of variance. Why do we have to set Lambda for it? [image]

How does regularization work on layer with activation "relu" in neural network?

Christian_Simonis January 4, 2023, 5:27pm 4

this thread touches upon ReLU vs. linear regression, too. I believe it could be interesting for you and should clarify potential doubts.

Have a good one!

Best regards
Christian

Topic		Replies	Views
How does regularization work on layer with activation “relu” in neural network? Improving Deep Neural Networks: Hyperparameter tun week-1	2	124	May 15, 2024
Tips on applying regularization AI Discussions	18	106	November 22, 2023
Questions about regularization Improving Deep Neural Networks: Hyperparameter tun week-1	6	30	July 13, 2024
Why using ReLU doesn't result in high bias error? Improving Deep Neural Networks: Hyperparameter tun	1	553	April 25, 2022
Week 1 - why regularization works with ReLu Improving Deep Neural Networks: Hyperparameter tun	5	665	January 14, 2022