W 3_A1_ReLU vs tanh accuracy

It’s great that you are trying experiments like this. You always learn something when you try to extend the ideas in the course. Here’s another thread related to this topic from a while ago. I was able to get 81% accuracy using ReLU with n_h = 40 and some other folks were able to get 85% accuracy with ReLU.

Your implementation of ReLU and ReLU’ look correct to me, but maybe you need a higher n_h value. Note that the n_h = 4 that works pretty well with tanh gives really terrible results with ReLU.

1 Like