Non-linear regression

[Christian_Simonis]

@TMosh

If the features themselves have non-linear characteristics, you will get a non-linear f_wb curve.

That’s a very good point: here a concrete example how to describe non-linearity in the features already:

Can we start with the circle equation as decision boundary?

Hi @tbhaxor, in addition to @rmwkwok 's excellent answer. I agree: Feature Engineering will do the trick: Of course you can also apply a transformation as we know it from polar coordinates to get rid of the non-linearity in the data and solve the problem with a linear classifier. [Bildschirm­foto 2023-01-14 um 20.36.23] import numpy as np import math import matplotlib.pyplot as plt # Some Parameters dim = 100 noise = 0.2 #Create inner circle x_1 = np.cos(np.linspace(0,2*math.pi, dim)) + …

One more point regarding this statement:

@TMosh

“non-linear regression” really isn’t a separate topic.

I am a huge fan of feature engineering but while it is often necessary, unfortunately it is not always enough to succeed: in reality only encoding non-linearity in features is not always sufficient to capture the full complexity of the relationship between variables.

Therefore in practice, non-linear models still play an important role when linear models w/ manual feature engineering get to their limits. You also see this when taking a look at the yearly publications and patents from top tier universities and enterprises.

One example is the lotka volterra sequence from my previous post where I believe you cannot build a sufficiently accurate solution modelled with a linear model and feature engineering only!

Best regards
Christian