Can logistic regression be replaced with ordinary linear regression


thanks for your question!

Regarding the statement:

the simple (non-logistic) linear regression will have advantage over logistic regression in the case of non-binary classification (for instance, when we have to classify onto three categories)

I think in general this statement is not true. As a counter example, let’s assume you want to build an early warning system to determine whether an information is correct or not [you could also think of an anomaly detection].

You could go for logistic regression, use the probability p for a false information and defined thresholds to derive a multi classification (similarily what you would do with a linear model as far as I understood), e.g. with the following logic:

  • p < 0.2 —> OK: seems to be fine
  • 0.2 \leq p < 0.7 —> suspection: some action is recommended like getting an expert review
  • p \geq 0.7 —> NOT OK: information seems to be wrong.

This approach might not necessarily perform worse than a linear regression model which you also would use for threshold based conclusions, right?

I would say the model suitability really depends on the business problem and how well a certain model fits a data set and can generalise on it.

Here some threads that you might find useful to check out:

As @rmwkwok pointed out, I would also underline:

it is not common to model a binary classification problem with a linear regression

Same applies also for a multi class problem:

Suggestion: In this case feel free to check out multi class algorithms. E.g. here you can find several really powerful models for that purpose.

Please let me know if this helps.

Все найкраще & all the best, Vasyl!

Best regards

1 Like