DualTTA: calibrating black-box radiology AI without model weights

Model-agnostic calibration for proprietary 3D CT models using test-time geometric perturbations and confidence-gated blending, cutting calibration error up to 54%.

Primary source: arXiv:2609.29931: https://arxiv.org/abs/2609.29931

What’s new: Commercial clinical AI models are typically proprietary black boxes: hospitals have no access to weights, gradients, or the pre-training cohort, making standard calibration methods (Temperature Scaling, MC Dropout, deep ensembles) impossible to apply. In Improving Calibration of Black-Box Radiology AI Using Test-Time Augmentation (arXiv:2609.29931, MICCAI 2026 UNSURE), researchers introduce DualTTA.

DualTTA perturbs input 3D CT volumes using clinically grounded geometric and physics-inspired transformations at test time. It then applies learned probability aggregation combining per-augmentation weighting with confidence-gated blending between the baseline prediction and the augmented ensemble. On pulmonary embolism detection, it reduced Expected Calibration Error (ECE) by 54% (0.239 down to 0.109); on intracranial hemorrhage detection, ECE fell by 43% (0.051 down to 0.029), outperforming internal-access baselines.

Why it matters: Well-calibrated risk estimates are required before clinical decision thresholds can be trusted. DualTTA shows that hospital IT and auditing systems can post-process third-party black-box vendor outputs to restore calibration without vendor cooperation or model retraining.

Caveats: Test-time augmentation multiplies inference compute by the number of perturbation passes, which may constrain real-time emergency triage workflows. The choice of 3D volumetric perturbations must be validated per anatomy to prevent artifact-induced misclassifications.