3

Based on answers to this question, I should be able to build a random forest with all the same trees by using bootstrap = False, max_features = None, random_state = 42 parameters.

I wrote quick code to test it, and it seems that different trees are created.

Is it possible to create a random forest using RandomForestClassifier which will produce the same trees?

Ethan
  • 1,633
  • 9
  • 24
  • 39
Damian Melniczuk
  • 619
  • 4
  • 19
  • Why do you want them all to be the same? – kbrose Jul 05 '18 at 13:45
  • it's right there in the accepted answer to the linked question if you read the entire answer – oW_ Jul 05 '18 at 15:27
  • 1
    @kbrose - as an example step in process of creating decision tree, looking for its hyperparameters, creating homogenous forest (with those hyperparameters), adding bootstrap, adding fraction of features and increasing n_estimators. So such forest would be something different than single tree, but with exactly the same predictive capability. – Damian Melniczuk Jul 06 '18 at 05:48
  • @oW_ Yes, I read the entire answer, especially You can fix this by passing the same seed to each DecisionTreeClassifier which you can do using random_state=something. RandomForestClassifier also has a random_state parameter which it passes along each DecisionTreeClassifier. - I did that, and it did produced trees with fixed seed but different for each one. So it seems that there is no such possibility? – Damian Melniczuk Jul 06 '18 at 05:51
  • 1
    this homogenous forest would not be different from a single decision tree. If they all share the same random seed then the randomness introduced by bootstrap and max_features would disappear ... that's exactly the point... that's also the reason why you can't do it with RandomForestClassifier..unless I'm missing something here – oW_ Jul 06 '18 at 15:22
  • Agreed oW_. If all trees are exactly the same you don’t have a forest, you have a tree. If the trees are not exactly the same then your forest has more predictive capabilities than a single tree. Maybe I’m not understanding something crucial here. – kbrose Jul 12 '18 at 13:56

1 Answers1

4

You want to create a random forest where all decision trees are the same. (I am trusting you are trying to do this as an educational exercise, not as something serious. :))

You tried to do this by setting random_state and cited my answer from

Indeed, as you discovered, my answer is not fully correct.

I have edited it to add

While this removes the stochasticity component of the training, the decision trees would still be different. The thing is that sklearn ensembles generate a new random seed for each child based on the random state they are given. They do not pass along the same random_state.

You can see this is the case by checking the _set_random_states method from the ensemble base module, in particular this line, which propagates the random_state across the ensembles' children.

As you can see, sklearn uses the random_state you give RandomForest to create new random states for each child,

for key in sorted(estimator.get_params(deep=True)):
    if key == 'random_state' or key.endswith('__random_state'):
        to_set[key] = random_state.randint(MAX_RAND_SEED)

Therefore they will be different. This is done presumably because heterogeneity is what makes an ensemble powerful. Notice this is not specific to RandomForest, but to any sklearn ensemble.

Sorry for my incorrect answer to the question you cite. If you want, you can use the random forest implementation. That one is completely homogeneous.

Ricardo Cruz
  • 3,410
  • 1
  • 15
  • 34