Hi @kaki178925
Sometimes fitting a streight line or a plane (w/ a linear model) is just not sufficient to model the relationship between your features and your label, especially if the relationship is a bit more complex.
In order to describe the non-linearity, feature engineering can do the trick which can involve:
- different powers of x as you mentioned
- features crosses, see also this thread: Example of encoding the non linearity using feature crossing - #2 by Christian_Simonis
- or some other kind of domain knowledge that you model into your features which is called feature engineering
When you have a manageable amount of features what always helps is to analyze the residuals to see where you have still systematic patterns left that you could potentially exploit in your feature engineering, see also:
- this thread: How to evaluate accuracy of a regression model - #21 by Christian_Simonis
- a coding example
In a perfect world you would just see some random (Gaussian) distribution in your residua and no systematic patterns:
Since your question also touches upon logistic regression, maybe this thread could be interesting for you, too: Can logistic regression be replaced with ordinary linear regression - #16 by Christian_Simonis
Hope that helps!
Best regards
Christian