TabBench-Bio: evaluating ML where biomedicine actually lives — wide tables, tiny n

A living eval suite of 43 biomedical HDLSS datasets across 28 feature×sample operating points — tabular foundation models lead at the reference cell, but logistic regression stays uncomfortably close.

Primary source: arXiv:2609.07441: https://arxiv.org/abs/2609.07441

What’s new: Gene-expression matrices, methylation arrays, and metagenomic profiles routinely offer thousands of columns and tens or hundreds of labels. General tabular benchmarks barely cover that HDLSS (high-dimensional, low-sample-size) geometry. TabBench-Bio (arXiv:2609.07441, Kreuer et al., submitted 7 Sep 2026) is a living benchmark of 43 biomedical datasets under a shared cross-validation protocol, comparing classical estimators, neural nets, and tabular foundation models across a 4×7 feature-by-sample grid (28 operating points).

At the reference cell of 10,000 features / 100 training samples, RealTabPFN v2.5 has the highest point estimate, then Logistic Regression and TabDPT with nearly identical point estimates. A paired bootstrap over the target pool separates RealTabPFN v2.5 from Logistic Regression by 145 Elo (95% interval [59, 232]). TFMs generally occupy the leading ranks, but the strongest configuration depends on operating point and modality. AutoGluon’s one-hour “extreme” preset is reported separately as a resource-heavy AutoML reference. Fold-level predictions and deterministic aggregations are released for reuse; interactive leaderboard: tabbench-bio.eu.

Why it matters: Biomedical tabular ML is where a lot of translational work actually happens, and “who wins on TabArena” does not answer “who survives 10k features and n=100.” Making the feature×sample grid part of the benchmark definition — not a footnote sensitivity plot — is the eval contribution. The awkward neighbor result is also useful: logistic regression remains competitive enough that any TFM victory needs an uncertainty interval, not a vibes ranking.

Caveats: Preprint / living benchmark — rankings will move as datasets and models are added. Elo gaps are on the authors’ target pool and protocol; the coarser target-level win rate for RealTabPFN vs logistic is less decisive (their abstract-level bootstrap on Elo excludes zero, but win-count intervals can still include a coin flip depending on aggregation — check the paper’s dual reporting). Fixed configurations emphasize robustness over heavily tuned peaks. AutoGluon’s extreme preset is a different compute budget, not an apples-to-apples peer. Use TabBench-Bio to pressure-test HDLSS claims; do not treat one reference cell as “TFMs solved biomed tables.”