Decision Trees for Regression?

I just completed week 4 specifically regarding decision trees and appreciate the material. One question I have is how are decision trees a good choice for regression problems? I can see how they perform well for classification problems but I just don’t understand how a decision tree would come down to a floating point value prediction if you have a regression problem? Thank you.

As far as i know is classification and tabular data!

Here is an example:

1 Like

Thanks Tom I appreciate the link. Is it fair to say that neural networks are better for regression problems in general?

General rules are seldom reliable. It depends on what you mean by “better”.

1 Like

I missed the “regression” word here, thanks for clarifying!

Personally, I have had really good experience with using decision trees for regression tasks, especially with bagged ones, in particular random forests. After all, they are quite well to interpret. Explainability is also good and with structured data (which is not super huge) and a limited amount of features, in which you can model your domain knowledge, they are a really good alternative or benchmark to Gaussian Processes in my opinion.

Also boosted trees work also often well.

I believe these threads might be relevant:

Happy learning!

Best regards

Nope, I do not agree with that statement in general, see also these sources, where I explain why:

The tendency in my opinion is:

  • the bigger the data
  • the more unstructured the data (videos, images)
  • the less domain knowledge you can encode in features
  • the more freedom or capacity your model needs to abstract really complex patterns

→ the stronger the benefits of deep neural networks with modern architectures and advanced layers are compared to traditional or classic ML. Also, when training deep neural networks you can leverage modern digital infrastructure like GPU clusters to accelerate the training.

Basically you have two approaches:


Hope that helps, @Amit_Misra1. All the best! :rocket:

Best regards