Non-linear regression

That’s a very good point: here a concrete example how to describe non-linearity in the features already:

One more point regarding this statement:

I am a huge fan of feature engineering but while it is often necessary, unfortunately it is not always enough to succeed: in reality only encoding non-linearity in features is not always sufficient to capture the full complexity of the relationship between variables.

Therefore in practice, non-linear models still play an important role when linear models w/ manual feature engineering get to their limits. You also see this when taking a look at the yearly publications and patents from top tier universities and enterprises.

One example is the lotka volterra sequence from my previous post where I believe you cannot build a sufficiently accurate solution modelled with a linear model and feature engineering only!

Best regards