I am curious about feature scaling and expansion. Scaling seem clear to me where data with greatly different values create challenging data plots that will make convergence take a long time or maybe impossible.
However what about feature expansion where you add features which could be the product of different features or the square or cubic value of a feature which is part of the feature vector. How does one know when to use feature expansion?
Can you perform both scaling and expansion together or are there more advanced techniques (learned later) that address problems where both are required?
Yes you could perform both together, if you know what you are doing!
For example, in physics, acceleration = Force/Mass, so instead of having 2 features, you only have one now. Now you have to study the data and see how they are related with each other and their impact on the output, you have to run some âdescriptive statisticsâ, see relations, trends before you use them in the model. I think there are platforms that can help with this process.
Maybe some of the data has no contribution to the output; hence no need to use it.
If you canât get a good fit to the data using the original features, then you can add new non-linear combinations of the original features.
This lets the model learn more complex relationships.
As Mentor Gent mentioned, the case of d = 1/2 a*t^2 (distance given time and acceleration) is a great example. if you only have measurements of d vs. t, you wonât get a good model to predict âdâ unless you create a new t^2 feature.