Hello,
In the Demand Prediction Video at timestamp 12:33, Andrew mentions that Neural Networks can learn its own features and not manually adding features. The example is a non-linear feature multiplying frontage (x_1) and depth (x_2). Doesn’t the neuron only perform linear operations? Doesn’t that mean the feature (like this one) must still be engineered?
Thanks,
RK
Hello RK @rmkalluri,
Firstly, while x_1 times x_2 is a helpful feature (let’s call it “the size”), it does not mean that the neural network has to produce exactly it as any one of its intermediate outputs to make the network a good model. In other words, an optimal model may not require “the size” at all.
Secondly, while it’s true that a neural network layer performs a linear combination of its inputs, the output of a layer or the output of multiple layers can become non-linear with respect to the features by adding non-linear activation functions. Therefore, as long as you give it enough nodes, layers and activation functions, it would still be possible to produce “the size” or other non-linear combinations of the features.
Cheers,
Raymond
In linear regression, yes.
But in a NN, the hidden layer has a non-linear activation function.
I see. Thank you for the clarifications.
So, there are different kinds of activation functions. Is a typical NN designed with a certain activation function? Or, is it intentionally overfitted (allowing multiple activation functions) and regularization applied to remove those functions that don’t seem useful?
No, that’s not what Regularization is used for.
Typical NN hidden layer activations (which you will learn more about in the Deep Learning Specialization)