In the final lab of MLS Course 1 Week 3, one of the steps uses feature mapping to increase the dimensionality of the classifier. It adds 25 dimensions, up to the 6th power.
I’d like to understand more about how this works. How do you decide how many additional dimensions to add, and up to what power? This feels like an essential step for being able to do some basic regression analysis on your own.
Experimentation. Add only as many features as are necessary to get good enough results,
Is there a recommended way to develop your intuition here? Is it pure trial & error?
If you are doing linear regression, you are assuming your features to be linear with the outcome. So, one way to verify in prior how likely your feature and engineered feature will work, is to make feature-outcome scatter plot for each feature, and see how much it looks like a linear relationship.
Experimentation is always the valid way.
So for example, if you are doing prediction for the following graph:
Then in order to get the best polynomial model to fit, you need to keep adding and test different combination of number of features and number of powers, correct? And most likely comes from experiences