Robust contextual MDPs optimize adaptive cancer therapy under biomarker and dynamic uncertainty

Formulating adaptive therapy through robust contextual reinforcement learning yields treatment schedules that Pareto-dominate clinical heuristics and maintain efficacy despite noisy biomarker measurements.

Primary source: arXiv:2610.22336: https://arxiv.org/abs/2610.22336

What’s new: In metastatic cancer management, continuous maximum tolerated dose chemotherapy often accelerates treatment failure by eradicating sensitive cells and selecting for aggressive, drug-resistant phenotypes. Evolutionary adaptive therapy aims to maintain competition between sensitive and resistant clones by titrating drug administration. However, existing clinical protocols (such as the standard AT50 regimen for abiraterone therapy in metastatic prostate cancer) rely on rigid heuristics, such as halting therapy when prostate-specific antigen (PSA) drops by 50% and resuming when it returns to baseline. While reinforcement learning (RL) could discover personalized dosing policies, clinical simulators suffer from severe patient heterogeneity and noisy biomarker assays. In Robust Reinforcement Learning for Dynamic Treatment Regimes under Model and Measurement Uncertainty (arXiv:2610.22336, 15 Oct 2026), researchers Chengze Xie, Xuan Wang, Ying Ding, Hairong Wang, and Yan Li formulate adaptive therapy through the mathematical framework of robust contextual Markov decision processes (R-CMDPs).

The authors treat each patient as a hidden context vector governing tumor cell subpopulation proportions and growth kinetics. By framing policy optimization as an R-CMDP, the algorithm accounts for both between-patient parameter variability and in-patient measurement perturbations in PSA observations. The authors establish finite-sample generalization bounds on the number of patient contexts required to learn provably near-optimal policies. In rigorous numerical evaluations using a biophysical prostate cancer simulator fitted to real-world clinical trial and cohort data, the resulting R-CMDP policies Pareto-dominated both clinical AT50 and standard non-robust RL methods. Under nominal conditions without observation errors, the robust policy achieved prolonged time-to-progression matching or exceeding standard RL, while under substantial PSA measurement noise (up to 30% relative error) and unexpected shifts in tumor proliferation dynamics, the policy preserved treatment efficacy without triggering premature tumor flare-ups.

Why it matters: Dynamic treatment regimes in oncology must tolerate high clinical uncertainty: laboratory biomarker measurements fluctuate, tumor microenvironments evolve, and no two patients share identical drug response curves. Non-robust RL policies easily overfit to synthetic simulator dynamics, producing brittle schedules that fail under real-world noise. By providing rigorous generalization guarantees and demonstration of noise resilience in calibrated clinical simulators, this work demonstrates how robust contextual RL can design dependable, patient-specific adaptive dosing protocols.

Caveats: The experimental evaluation is conducted in silico using three-population Lotka-Volterra ordinary differential equation models calibrated to abiraterone trial cohorts; real-world biological tumors may exhibit higher-order spatial heterogeneity and mutation mechanisms outside the modeled state space. Furthermore, the action space considers discrete on/off dosing decisions at monthly intervals rather than continuous dose titration, and clinician acceptance will require interpretable safety constraints and prospective observational pilot studies before bedside deployment.