Sparse autoencoders detect imminent robot policy failure in OpenVLA before physical collapse

Rogers, Lin, and Brown apply sparse autoencoders to OpenVLA activations, identifying interpretable latent features that predict manipulation failures before irreversible physical errors occur.

Primary source: arXiv:2610.32208: https://arxiv.org/abs/2610.32208

What’s new: Vision-Language-Action (VLA) models provide unified policies that translate visual observations and natural language goals into physical robot control. However, diagnosing why a VLA policy fails remains opaque: end-to-end action chunks offer little visibility into whether a failure stemmed from misinterpreting the instruction, misjudging spatial affordances, or motor drift. In Do VLAs Know They Will Fail? Interpretable Success and Failure Patterns in Physical AI Systems (arXiv:2610.32208, 23 Oct 2026), Christian A. Rogers, Qian Lin, and Daniel S. Brown demonstrate that internal activations within OpenVLA reliably forecast task failure well before the robot reaches an unrecoverable state.

The authors train sparse autoencoders (SAEs) on the internal representation space of OpenVLA during robotic manipulation rollouts. Linear probes trained on SAE feature activations predict task failure early in the execution trajectory, outperforming probes trained on raw dense activations. Beyond early warning, the SAE dictionary decomposes policy computations into monosemantic, human-interpretable failure modes, such as premature gripper release, spatial overreach, or tool misalignment. Rogers and colleagues build an automated interpretation pipeline using an external vision-language model to generate behavioral explanations for significant activation shifts, verifying that feature activations correlate with specific physical failure stages across held-out evaluation episodes.

Why it matters: For autonomous robots operating in physical environments alongside humans or handling delicate industrial assemblies, reactive error handling after a collision or dropped object is unacceptable. Showing that open-weight VLA models harbor internal latent indicators of impending failure provides a practical mechanism for runtime safety governors. Robots can monitor their own SAE representations to pause execution, trigger safe fallback routines, or request human intervention before causing physical damage.

Caveats: The predictive accuracy of early failure detection degrades on out-of-distribution visual scenes or unfamiliar object geometries that deviate substantially from the SAE training distribution. Moreover, running sparse autoencoder transformations alongside real-time VLA action token generation adds runtime memory and compute overhead, which may challenge compute-constrained edge robot controllers.