EnsembleSample
Random forest
Averages many decision trees, each trained on a random sample of the data.
Synthetic data, generated and trained in your browser.
Try this
- 01Set 1 tree: the boundary is as blocky as a single decision tree.
- 02Slide to 60 trees and watch the boundary smooth out.
- 03Compare with the decision tree on XOR with the test split on.
How it works
- 1Draw a bootstrap sample (with replacement) of the training data.
- 2Grow a tree on it, choosing from a random subset of features at each split.
- 3Repeat for many trees.
- 4Average their votes: individual trees overfit, but their errors partly cancel out.
What the controls teach
- Trees
- Number of trees averaged. One tree is blocky; many trees give smoother, more graded regions.
- Max depth
- Depth of each tree. Forests tolerate deep trees because averaging reduces variance.
Use it when
- A strong default for tabular data
- Little time for tuning
Avoid it when
- Very high-dimensional sparse data
- When a single, readable rule set is needed