NPC-RubricBench reveals medical LLM diagnostic competence fails across longitudinal cancer care

Zhang et al. introduce NPC-RubricBench, evaluating 30 language models across longitudinal oncology care pathways and exposing critical failure mode shifts from diagnostic commission to treatment omission.

Primary source: arXiv:2610.35847: https://arxiv.org/abs/2610.35847

What’s new: Evaluating large language models on single, isolated clinical vignettes fails to reflect the complexity of longitudinal patient management. In clinical oncology, care involves sequential stages ranging from initial workup and diagnostic staging to treatment planning, response monitoring, and recurrence surveillance, where clinical priorities shift continuously. In Patient-specific rubric-based evaluation of large language models across longitudinal cancer care (arXiv:2610.35847, 26 Oct 2026), Wei-Wei Zhang, Pengkai Wang, Yan Li, Zhijie Sang, Guan-Qun Zhou, and colleagues introduce NPC-RubricBench, a phase-aware benchmark tracking multistage clinical reasoning across the cancer care continuum using nasopharyngeal carcinoma as a primary model.

Curated from longitudinal medical records of 405 patients across two major cancer centers, the benchmark comprises 941 patient-phase units spanning six sequential care phases and 17,688 patient-specific atomic criteria across five clinical evaluation dimensions. Every evaluation strictly limits available information to decision-time evidence to prevent lookahead bias. Evaluating 30 frontier general-purpose and specialized medical LLMs, the authors uncover a striking disconnect: high performance on initial diagnostic tasks fails to carry over to downstream management. Furthermore, the dominant failure modes invert across the care pathway: explicit commission errors dominate anatomical staging (48.2%), whereas omission errors dominate initial workup (33.4%) and post-treatment follow-up (30.4%). Surprisingly, medically adapted models exhibited higher omission rates than general-purpose frontier models. The authors also identify an acute prompt specification trade-off: broad open-ended prompts degrade decision quality across all 30 models, while detailed checklist-style prompts improve decision quality at the cost of evidence faithfulness and safety adherence.

Why it matters: Conventional medical AI benchmarks rely heavily on static multiple-choice questions or isolated consultations, creating an illusion of clinical readiness. NPC-RubricBench demonstrates that clinical reasoning is not uniform across a disease trajectory: an LLM that stages a tumor accurately may still systematically omit critical monitoring labs or recurrence checks. Establishing phase-adaptive guardrails and evaluating models across longitudinal care pathways is essential before deploying LLM assistants into real-world hospital workflows.

Caveats: The benchmark focuses specifically on nasopharyngeal carcinoma across two cancer centers, and its findings may reflect domain-specific oncological protocols that do not generalize uniformly to other cancer types or chronic non-malignant diseases. In addition, the atomic criteria were derived from retrospective electronic health records, which may not capture informal clinician-patient communication or real-time diagnostic nuances.