in the above picture, the lab says that unit 1 influence unit 2 but unit 0 doesn’t influence anything , can anyone please explain how that works?
The outputs of all of the ReLU units are added together.
The first segment (Unit 0) works because the weight value for that unit is negative. So it’s output has a negative slope. It’s output is activate for x values below 1.0.
2 Likes
Thank you sir for the reply , so in this example if the unit 0 slope was positive it will affect unit 1 and unit 2 as well?
Hello @Abbas_Mohammed
The ReLU gives a non-zero ouput when the input > 0. For all other input values the output of ReLU will be 0.
Now, look at the plots of z for unit 1 2 and 3. The pink line a gets activated from the point that z > 0 for that particular unit - This applies for units 1, 2 and 3.
Add up the a values from unit 1, 2 and 3 and you will get the plot of the Target on the left.
To summarize, a gets activated when z > 0. The a of each unit contributes to the output from the point that z > 0 for that unit.
2 Likes