in the first video of week 3, Luis shows a diagram of a neural network indicating the neurons are linear models and that the NN can model non linear systems; however, per the ML specialization, wouldn’t he have to use a non linear activation function (such as sigmoid or ReLU) in order to model non linear behavior? If all the neurons are just the normal linear activation function, my understanding is only linear behavior can be modeled (because a linear equation of a linear equation will still be linear)
No.
The key in a NN is that the hidden layer contains a non-linear activation.
In the example he showed though, I believe he only used a linear activation function in all the hidden layers:
unless something was different in the output layer (sigmoid, ReLU), wouldn’t this be linear?
There is an error in the diagram (but that’s not usual for the M4ML course). At least one of those columns of “linear model” must be “non-linear activation”.
2 Likes