Branchwise Linear/Polynomial Regression

[codingninja007]

This is the article on polynomial linear regression for reference:

Polynomial regression - Week 2: Regression with multiple input variables |...

Is there any method to split a training set to, say, two branches, and separately apply linear/polynomial regression to both? According to my intuition, logistic regression could be of some help here but I am confused as to how to implement it?

One use case of this could be simulating piecewise continuous polynomial functions which are not necessarily differentiable. In such a case, clearly, one polynomial will not minimize the cost.

[gent.spah]

I think you will learn more about this as you go on the specialisation!

You dont even need to split it, you can use the same training dataset with different models, either be it polynomial, logistic or even neural network and see which one performs better for the dataset, but there are other factors to consider as well!

[TMosh]

@codingninja007

Is there any method to split a training set to, say, two branches, and separately apply linear/polynomial regression to both?

Typically this is not necessary, as it is complicated and would require human intervention to decide where to make the splits.

We’re doing machine learning, so we want to minimize the amount of human expertise required to work with the data set.

[rmwkwok]

Hello @codingninja007

You were saying to setup a few models on different ranges of the X axis? Then, the method you are asking for is for automatically finding the split points?

Realizing that the split points are actually part of the model, finding the split points is just a sub-task of learning model parameters, which means that we need an approach that can model split points, and polynomial regression is definitely not the right one.

A good one, that we will learn in this MLS Course 2, is just neural network with ReLU as activations. You will see that ReLU is a piecewise-linear function, and a Neural network behaves just like joining up some piecewise-linear ReLU to form a piecewise-linear “curve”.

Piecewise-linear curve is a good way for your problem, because each polynomial can be approximated by a piecewise-linear curve. In other words, if a polynomial is approx’ed by a piecewise-linear curve, a piecewise-polynomial can then be approx’ed by a longer piecewise-linear curve.

In short, the method you are looking for could just be neural network that models piecewise functioin with ReLU. Taking this specialization means that you are already on the right track to it!

Btw, if I knew what kind of polynomials are there, I would create them and add them as polynomial features for training the neural network model.

Cheers,
Raymond