Week2 relu lab

[TMosh]

Found it:

C2_W2_Relu activation lab

I have been curious to learn the details of exactly how this works, so I did an experiment. I created a training set that is a parabolic curve, where x goes from -5 to +4, and y = x^2. I set up a 2-layer NN, with one input unit, 5 hidden layer units with ReLU activation, and one output unit with linear activation. It converged nicely, and here is a plot of ‘y’ and ‘y-hat’. [image] Here’s what is inside each ReLU unit: z = max(0, w*x + b) So each ReLU unit can optimize two values - the sl…