How does regularization work on layer with activation “relu” in neural network?

Joshua_Mawutor_Agban · May 15, 2024, 9:17am

Improving Deep Neural Networks: Hyperparameter tun WK-1 , regularization
Having error with my personal project. RELU activation affecting the cost function. any help
Regularization.py (6.2 KB)
TRY_MODEL.ipynb (8.5 KB)

I need help. Thank you

gent.spah · May 15, 2024, 10:08am

Relu clips the negative weights so regularization would reduce the positive weights of the relu layers if needed!

Joshua_Mawutor_Agban · May 15, 2024, 1:27pm

Please can you explain it for me!! and if possible, kindly run the python files for me and make any recommendadtion for the errors. Thank you

Topic		Replies	Views
How does regularization work on layer with activation "relu" in neural network? Advanced Learning Algorithms week-3	3	650	January 4, 2023
Questions about regularization Improving Deep Neural Networks: Hyperparameter tun week-1	6	36	July 13, 2024
Neural network regularization: What does it mean to regularize a hidden layer? Advanced Learning Algorithms week-3	4	549	April 3, 2023
Week 1 - why regularization works with ReLu Improving Deep Neural Networks: Hyperparameter tun	5	686	January 14, 2022
C2 W1 / Regularization / Exercise 2 Improving Deep Neural Networks: Hyperparameter tun	2	562	February 15, 2022