Chain-of-thought monitoring can fail even when reasoning is strictly necessary

Counterfactual interventions across Qwen-3 and Gemma-4 show that attributes driving 19 to 24% of correct medical diagnoses go unmentioned in reasoning traces that models cannot solve without CoT.

Primary source: arXiv:2610.18105: https://arxiv.org/abs/2610.18105

What’s new: Safety oversight for reasoning models often assumes that previously documented Chain-of-Thought (CoT) faithfulness failures stem from post-hoc rationalization on easy tasks, and that CoT traces become reliably monitorable whenever a model genuinely needs multi-step reasoning to reach its answer. In Chain of Thought Monitoring Can Fail Even When CoT Is Necessary (arXiv:2610.18105, 13 Oct 2026), researchers from Fraunhofer HHI, TU Berlin, TU Dublin, and BIFOLD refute this binary assumption. They introduce a four-level hierarchy of CoT necessity: Level 0 (CoT unnecessary for the answer), Level 1 (CoT necessary to reach the answer, operationalized as no-CoT accuracy $p_{10} < 0.01$), Level 2 (CoT necessary to process the specific monitored property), and Level $3.m$ (requiring $m$ consecutive forward passes dedicated to the monitored property).

Evaluating Qwen-3 (4B, 8B) and Gemma-4 (E4B-it, 31B-it) via distributional counterfactual interventions (2,000 sampled reasoning traces per sample-attribute condition) across multi-class benchmarks including Thyroid Disease and DDXPlus Medical diagnosis, the authors isolate Level 1 regimes where models cannot solve the cases without CoT. Even in this regime, monitorability collapses ($M_1 \approx 0$) for key clinical features: on Thyroid Disease, patient sex and thyroxine history systematically determine the correct diagnosis in at least 19% and 24% of CoT-dependent responses, respectively, yet their influence is almost never articulated in the reasoning text. Causal mediation interventions that freeze intermediate CoT hidden states further reveal that direct answer-stage integration explains only 7% (patient sex) and 14% (thyroxine query) of the total causal effect, demonstrating that models integrate these features diffusely during the necessary CoT tokens (Level 2) without verbalizing them.

Why it matters: Frontier safety frameworks increasingly treat CoT monitoring as a primary defense against sandbagging, alignment faking, and biased clinical decision-making on hard tasks. Demonstrating that models can causally rely on sensitive attributes throughout necessary multi-step computations without leaving a verbal trace proves that task difficulty alone does not guarantee transparent reasoning.

Caveats: The study evaluates benign multi-class prediction tasks as controlled proxies rather than active deceptive behaviors such as sandbagging or sabotage. Because stochastic token sampling prevents labeling counterfactual influence on individual trajectories, the framework bounds monitorability over response distributions and omits a true-negative specificity score. Finally, the high computational cost of two-stage counterfactual sampling (requiring roughly 3,000 A100 and 2,000 B200 GPU hours) restricted evaluation to open-weight models up to 31 billion parameters.