Regression versus classification

[Sam_R]

Hello,
I was thinking about the distinction of classification versus regression.

1. What type of algorithm, amongst these two, would translation from one language to another be?
   It seems to me that it fits better a regression, since there is an unlimited choice of translation options. That is in the case of an advanced translator, whereas a naive translator which translates word-for-word (without taking much context into consideration) can use classification, imho.

2. It seems to be that the output of a regression model, can be any real number. Suppose that only a positive input actually makes sense in the context of our model (eg prediction weight from height). Would we gain anything as researchers by restricting, in some way, the regression model to output only positive numbers ?

[hackyon]

@Sam_R

• What type of algorithm, amongst these two, would translation from one language to another be?
  It seems to me that it fits better a regression, since there is an unlimited choice of translation options. That is in the case of an advanced translator, whereas a naive translator which translates word-for-word (without taking much context into consideration) can use classification, imho.

Neither. Normally, you won’t use classification or regression for translation tasks.

There are other models for that (covered later in MLS or DLS).

@Sam_R

• It seems to be that the output of a regression model, can be any real number. Suppose that only a positive input actually makes sense in the context of our model (eg prediction weight from height). Would we gain anything as researchers by restricting, in some way, the regression model to output only positive numbers ?

Yes, you should do this. The AI model may give you a pretty good prediction for regressions, but you should apply any known limitations and bounds to the output as necessary.

[TMosh]

@Sam_R

Would we gain anything as researchers by restricting, in some way, the regression model to output only positive numbers ?

I vote No on imposing artificial limits on the model.

You want to know how well the model is working. Artificially modifying the output will make it difficult to understand and improve your model.

[Sam_R]

@hackyon

Yes, you should do this. The AI model may give you a pretty good prediction for regressions, but you should apply any known limitations and bounds to the output as necessary.

Is there a way of doing it in linear regression (eg predicting weight from height)?

[rmwkwok]

Hello @Sam_R

The easiest way is to clip any negative predictions to 0.

If we want to do it at training and at all cost, which is obviously off the course’s syllabus,

• a soft way would be to consider it as an optimization problem and, in tensorflow for example, use a custom loss function that adds extra terms like
  Loss = OriginalLoss - \lambda y or
  Loss = OriginalLoss - \lambda ReLU(-y)
  , which will penalize negative y.

• the hard way would be to set up a tensorflow model of one layer & one neuron with ReLU as the activation, because ReLU prevents negative output. However, we no longer have a linear model.

Cheers,
Raymond