3

I have an understanding of how random forests can be used for regression. But, I am still unable to understand, how random forests can be used for regression. I have already read the research work of Leo Brieman on random forests. But, I still fail to understand how random forests can be used for regression.

It would be of great help, if this concept can be explained using any toy example.

Thanks in advance.

Swaroop
  • 161
  • 6
  • mods: should this be migrated to Cross Validated Stack Exchange? i only ask because this seems less like a signal processing question and more a machine learning question (although two will surely overlap more and more in time) – panthyon Sep 28 '15 at 23:56

1 Answers1

1

The key is in choosing the split function of the decision tree. In classification, the data is split according to a certain homogeneity metric. On the other hand, for a regression tree, a regression model is fit to the dependent variable, for each of the independent variables. Then, the data is split at multiple points and the SSD value is evaluated. The best split is chosen for the dimension (variable) giving the minimum error. The node is then split at this point. The splitting is again recursive.

Finally, each leaf stores a distribution for the continuous output variable/s. In the testing phase, the responses are averaged.

Tolga Birdal
  • 5,465
  • 1
  • 16
  • 40