EnsembleSample
Gradient boosting
Adds small trees one at a time, each correcting the errors left by the ones before.
Synthetic data, generated and trained in your browser.
Try this
- 01Press "Grow from 1 tree" and watch the curve assemble step by step.
- 02Set the learning rate to 1 with 150 trees and turn on the test split: the fit gets jagged.
- 03Compare 150 trees at rate 0.05 with 20 trees at rate 0.5.
How it works
- 1Start with a constant prediction: the mean.
- 2Compute the residuals: what the current model still gets wrong.
- 3Fit a small tree to those residuals and add a shrunken copy of it to the model.
- 4Repeat; the learning rate controls how much each new tree contributes.
What the controls teach
- Trees
- Number of trees added. More trees fit the training data ever more closely.
- Learning rate
- How much each tree’s correction counts. Smaller rates need more trees but generalise better.
- Tree depth
- Size of each tree. Depth 1 stumps make one cut each; deeper trees capture interactions.
Use it when
- Tabular data where accuracy matters most
Avoid it when
- Noisy data without early stopping
- When training time or interpretability is critical