HAD-VLM pairs autoregression with diffusion to preserve continuous geometric precision in medical imaging

Huang et al. introduce HAD-VLM, a unified vision-language framework combining autoregressive text with continuous diffusion prediction to eliminate quantization errors in clinical ultrasound and X-ray measurements.

Primary source: arXiv:2610.32948: https://arxiv.org/abs/2610.32948

What’s new: Vision-language models (VLMs) excel at qualitative medical reasoning, visual question answering, and report drafting, but struggle with precise geometric quantification. Because autoregressive language decoders discretize numbers into vocabulary tokens, prompting a VLM to estimate continuous physical dimensions (such as cardiac wall thickness or intervertebral disc spacing) introduces discretization error and hallucinated coordinates. In Beyond Discrete Tokens: Preserving Numerical Continuity in Medical Vision-Language Models (arXiv:2610.32948, 23 Oct 2026), Dianye Huang, Jiamei Guo, Yue Zhou, Mingde Li, Yi Xiong, Wei Wang, Nassir Navab, and Zhongliang Jiang present HAD-VLM, a hybrid architecture combining autoregressive text generation with a continuous diffusion head over a shared visual backbone.

HAD-VLM adapts a vision encoder using visual prompt tuning tailored to ultrasound acoustics. While text generation remains autoregressive, quantitative measurements are formulated as continuous five-component vectors: two pairs of caliper endpoint coordinates and the physical distance between them. A conditional diffusion process predicts this continuous representation, guided by language instructions. To enforce physical realism, the authors introduce a coordinate-length consistency constraint combined with ensemble prediction to refine final caliper placements. Evaluated on four echocardiographic measurement tasks on EchoNet-LVH (including interventricular septum and left ventricular posterior wall dimensions), HAD-VLM reduces mean absolute error by 21.4% to 37.6% compared to a Vicuna-based autoregressive baseline, outperforming fine-tuned Qwen3-VL and direct MLP regressors while preserving medical question-answering accuracy. On X-ray vertebral measurement, HAD-VLM slashes error from 11.4 to 6.7 pixels.

Why it matters: Clinical radiology and cardiology workflows require exact millimeter-scale measurements to diagnose pathologies like ventricular hypertrophy or aortic stenosis. Previous attempts to turn VLMs into end-to-end clinical assistants failed because tokenized generation cannot capture sub-millimeter geometric continuity. By wedding autoregressive prompt flexibility with diffusion-based continuous regression, HAD-VLM shows how foundation models can deliver rigorous geometric measurement without sacrificing conversational reasoning.

Caveats: The diffusion-based regression head requires multi-step iterative sampling during inference, increasing computational latency relative to direct single-pass coordinate regression. Additionally, the approach relies on explicit caliper coordinate supervision during training, limiting its immediate zero-shot applicability to imaging modalities or anatomical landmarks where annotated landmark coordinate pairs are unavailable.