Tabular Machine Learning.

2015–2022 · Trees rule

XGBoost (2016), LightGBM (2017) and CatBoost (2017) win essentially every tabular Kaggle competition and every fair benchmark. Repeated studies find deep learning fails to beat tuned GBDTs on typical tabular tasks. “Just use XGBoost” becomes the default advice.

2025 · The Nature moment

TabPFN v2 (Hollmann et al., Nature 637, 319–326) shows a transformer pre-trained on ~100 million synthetic datasets can outperform every baseline on datasets up to 10,000 samples × 500 features — in 2.8 seconds, beating ensembles tuned for 4 hours. Tabular ML gets its foundation-model era.

2025 · TabArena ships

The community gets a living benchmark (arXiv:2506.16791): 51 datasets curated from a pool of 1,053 across 14 prior benchmarks, 16 model families, Elo ratings with bootstrapped CIs, and a maintained public leaderboard. Its launch finding: validation protocol and post-hoc ensembling change rankings more than architecture choice.

2026 · Foundation models take the board

On the current board (v0.1.4), the top four single models are all tabular foundation models — TabPFN-3, TabPFN-2.6, RealTabPFN-2.5, TabICLv2. The best GBDT config sits ~240 Elo lower. Trees still win on CPU cost, large data, and operational simplicity — but they no longer set the accuracy frontier.

Small / medium data (≲50K rows) · max accuracy

TabPFN-3 · TabPFN-2.6 · TabICLv2

Foundation models hold the top single-model slots on TabArena, in one forward pass with no tuning. This regime — most real business datasets — is where in-context learning won.

Large data · CPU-only · production serving

LightGBM · CatBoost

Best Elo per CPU-second on the board. LightGBM (tuned + ensembled, Elo 1433) is the strongest tree; CatBoost's default config (Elo 1369, ~7 s/1K train) is the best zero-tuning classical pick.

Absolute ceiling · accuracy at any cost

AutoGluon (extreme preset)

The 4-hour multi-model ensemble tops the overall board (Elo 1695). If a Kaggle-style last percent matters and you have the compute budget, ensemble across families.

Regulated / interpretable

EBM · Linear + strong features

Explainable Boosting Machines are glass-box additive models — every prediction decomposes into per-feature curves. You pay ~160 Elo vs LightGBM for that property.

Deep-learning stack already in place

TabM · RealMLP

The two strongest trained-from-scratch nets (Elo 1447 / 1513 tuned + ensembled). They beat tuned GBDTs on the 2026 board — but only with tuning and post-hoc ensembling.

Honest baseline before anything fancy

Logistic / linear regression

Elo 962 tuned. If your candidate model doesn't clearly separate from this, invest in features and data quality, not architecture.

The official living benchmark for tabular ML (paper: arXiv:2506.16791). Datasets curated from 1,053 candidates across 14 prior benchmarks; every model run with defaults, tuned, and tuned + post-hoc-ensembled protocols; Elo calibrated to default Random Forest = 1000. The single board to read first in 2026.

The peer-reviewed evaluation behind the foundation-model turn: TabPFN outperforms all baselines on small data, with reported speedups of 5,140× (classification) and 3,000× (regression) versus 4-hour-tuned ensembles. Hollmann et al., Nature 637, 319–326.

The subset where the GBDT-vs-foundation-model question is most decisively settled: TabPFN-3 (Elo 1642) ties the 4-hour AutoGluon reference pipeline and leads the best GBDT config by ~250 Elo. If your datasets are small, this is the slice that predicts your experience.

Pichler, Salinas et al. — the methodology behind the board: why prior tabular benchmarks disagreed (dataset licensing, leakage, weak validation protocols), how post-hoc ensembling reshuffles rankings, and the maintenance protocol that keeps the leaderboard living.