Feature generation

Let me please ask a question.
In one of introductory videos Prof. Andrew shows how we go from input parameters (size, bedrooms, zip, wealth) to a new set of features (family size, affordability, school quality). I cannot understand the intuition behind creating these new features… Why could not we train the NN with initial set of features?
Thank you.
Kyiv, Ukraine.

Hi @vasyl.delta,

The more features you have, the better your prediction will be.

Talking about the house price prediction, let’s say your features only include number of bedrooms. This model will always give you the same price for, lets say, 3 bedroom house.

Let’s say you have another model and this time in the features, you include number of bedrooms and bedroom size. This model will not give you the same price for 3 bedroom house as it will also consider size of the bedrooms. More bedroom size, more price and vice versa.

As humans, we know, bigger bedroom size means more price so that’s what we are trying to capture…more information by adding more features for making better predictions.


1 Like

Thank you for the answer!
The question was rather on how to find new features…
Besides, In the mentioned example the input features remained the same, but the new ones were introduced in the next layer of neural network.

Basically you have two approaches:

There are several reasons why feature engineering is key. Couple of them are:

  • performance: e.g. you can model non-linearity already in your features and make it easy for your ML model for fit the data well. So whenever you can model „distilled knowledge“ in features this is valuable for modelling

  • costs: often this also helps you in terms of requiring less data for a good model which often reduces your costs, especially when labels are expensive to create

  • interpretability: often features have a nice domain interpretation (e.g. a physical meaning or an aggregation of knowledge) which also helps for plausibility checks and interpreting (or explaining model results to your stakeholders).

Hope that helps, @vasyl.delta.

Best regards

Thank you very much for the detailed answer!
Just to make one point a little bit clearier:
In the task of digit recognition we have a set of pixels with their colours as an input to the NN.
In the next layer these values are modified so to detect different parts of the digits. How do we come to the idea that this layer should extract parts of the digits? Has it to be decided by the engineer or DL algorithm itself can “understand” that such features are reasonable (because sometimes it is said that DL eliminates the stage of feature extraction)?
I humbly hope that I expressed my question clearly.