BatchBench evaluates AI robustness against severe batch effects in high-throughput cell painting

A standardized benchmark spanning 13 imaging sources and 2,000 compounds shows that while test-time adaptation mitigates domain shift, AI models degrade to chance level on rare phenotypic profiles.

Primary source: arXiv:2610.30104: https://arxiv.org/abs/2610.30104

What’s new: High-content microscopy assays such as Cell Painting capture rich morphological profiles of cellular responses to chemical treatments, serving as a pillar of modern AI-driven drug discovery. However, differences in laboratory environments, reagent batches, and optical imaging instruments introduce severe technical batch effects that confound machine learning models. In BatchBench: Benchmarking AI Robustness to Batch Effects and Domain Shift in Biomedical Images - A JUMP Cell Painting Leaderboard (arXiv:2610.30104, 21 Oct 2026), Sonja Topf, Ana Sanchez-Fernandez, and Günter Klambauer from Johannes Kepler University Linz introduce BatchBench, an open benchmark and leaderboard designed to systematically assess AI generalization under real-world biological domain shifts.

Built on the massive JUMP-CP (Joint Undertaking in Morphological Profiling - Cell Painting) dataset, BatchBench compiles approximately 140,000 high-resolution cell images across 2,000 compounds and 13 distinct imaging sources. The benchmark measures how accurately models identify chemical perturbations when evaluated on imaging batches from institutions not seen during training. Evaluating multiple paradigms, the authors show that standard supervised ResNet18 models achieve only 17.5% overall accuracy under domain shift. Test-time adaptation via Adaptive Batch Normalization (AdaBN) improves overall accuracy to 24.4%, but this improvement is heavily concentrated on high-data positive controls (POSCON accuracy rises from 46.5% to 64.6%). In contrast, domain-adversarial neural networks (DANN) fail to outperform unadapted models (17.0%), and on compounds with limited training samples (the REST subset, averaging 12 images per compound), all evaluated methods plummet to near chance levels.

Why it matters: Pharmaceutical pipelines increasingly rely on computer vision models to screen compound libraries and identify drug mechanisms of action across multi-center consortia. If biological representations fail to generalize beyond the specific well or plate where compounds were imaged, automated phenotypic screening risks false leads or missed therapeutic candidates. BatchBench establishes the first standardized testbed to track progress in domain-generalizable representation learning for cellular microscopy.

Caveats: The current benchmark focuses on single-cell phenotypic classification across 2,000 reference compounds; scaling evaluation to whole-well aggregate profiles and unannotated combinatorial drug screens will require further expansion. In addition, test-time adaptation methods like AdaBN require access to an entire test batch of images before prediction, which may not always be available in continuous single-well screening workflows.