Better Activation functions: (tanh > sigmoid)

[Osama_Saad_Farouk]

@rmwkwok @RR5555
Here is the thread I told you about:

Better Activation Functions (part 2)

In my last thread, [you can check it here] (Better Activation functions: (tanh > sigmoid)). I talked about why tanh is better than sigmoid in the hidden layers of neural networks. Today I’m going to share with you all about the Relu activation function and its variants. So, what is the Relu activation function? The Rectified Linear Unit (Relu): is the most commonly used activation function in deep learning models. The function returns 0 if it receives any negative input, but for any positive va…