Random Forest
Hundreds of deliberately mediocre trees, averaged — the strongest default in machine learning, and almost impossible to misuse.
When not to use it
- When you need to explain the decision. Five hundred trees is not an explanation, and post-hoc attribution is a story about the model, not the model.
- On images, text, or audio. These need learned representations. Trees operate on features you already have.
- On very large data where training time matters. Boosting is more efficient per unit of accuracy.
- On very wide, sparse data. Linear models handle text-like features better.
Reach for something else instead
- Gradient boosting — a few percent better, a day of tuning, less forgiving.
- Single decision tree — when the model must be readable.
- Logistic regression — interpretable, and competitive more often than people expect.
- Neural networks — for anything perceptual, or very large data with structure to learn.
Sources & further reading
- Breiman (2001), Random Forests — the paper; still clear, still worth reading directly.
- Grinsztajn et al. (2022), Why do tree-based models still outperform deep learning on tabular data? — the structural explanation, not just the observation.
- Fernández-Delgado et al. (2014), Do we Need Hundreds of Classifiers to Solve Real World Classification Problems? — 179 classifiers, 121 datasets; random forests came out on top overall.
Primary sources, listed so you can check the claims on this page rather than take them on trust.
Where people go wrong
- Tuning n_estimators. More trees never hurt accuracy; you're tuning your patience.
- Turning off feature subsampling. It's the mechanism that decorrelates the trees — without it the forest is just slower.
- Ignoring OOB error and building a separate validation set you didn't need.
- Using default impurity importance to explain the model. It's biased toward high-cardinality features. Use permutation importance.
- Reaching for deep learning on tabular data because it's modern. The evidence says otherwise and the reasons are structural.
At a glance
FieldMachine Learning
Ideaaverage many decorrelated trees
Two randomness sourcesbagging + feature subsampling
Ceilingtree correlation, not tree count
Free evaluationout-of-bag error
Reputationhardest model to misuse
DifficultyBeginner
Flashcards for this concept
Question
—
Answer
—
1 / 4
Often compared with
Random forest vs. gradient boosting — parallel and forgiving vs. sequential and sharper. The forest is the better default; boosting is the better result if you'll do the work.