The original assignment uses tanh activation for the first layer. I replaced it with relu activation. However, the model doesn’t want to learn. To do that I downloaded the notebook and made adjustments on my local environment. This is what I used for relu activation: def relu(x): return np.maximu…

When you use ReLU, you must increase the number of units in the hidden layer to replace a tanh() unit. This is because tanh() is a complex smooth curve. To get the equivalent performance from ReLU, you’ll need perhaps 10 to 20 units.

W3 A1 Relu Activation doesn't work

Course Q&A Deep Learning Specialization Neural Networks and Deep Learning

paulinpaloalto October 29, 2024, 1:23am 3

Here’s a thread from a while back in which using ReLU for this exercise is also discussed.

Here’s a post from mentor Raymond which goes into more detail on this question.

1 Like

Topic		Replies	Views
Course1 - Week3 Assignment - ReLU gave worse performance than tanh Neural Networks and Deep Learning	3	547	September 9, 2021
W3_A1_ReLu as Activation function Neural Networks and Deep Learning	3	622	March 30, 2023
How to apply relu function in Exercise of week 3(optional).) Neural Networks and Deep Learning	5	538	July 12, 2023
Using ReLU instead of tanh in DLS 1 week 3 Neural Networks and Deep Learning	1	493	March 15, 2022
W 3_A1_ReLU vs tanh accuracy Neural Networks and Deep Learning	8	637	November 13, 2022

W3 A1 Relu Activation doesn't work

Related topics