CarryoverBench exposes persistent prompt injection risks across multi-session LLM agent runtimes

Evaluating cross-session payload retention across frontier models reveals that malicious prompts planted in agent memory and heartbeat files re-fire without decay, bypassing conventional safeguards.

Primary source: arXiv:2610.28425: https://arxiv.org/abs/2610.28425

What’s new: Autonomous AI agents increasingly rely on persistent state across interactive sessions, utilizing long-term memory files (such as MEMORY.md) prepended to system prompts or scheduled heartbeat triggers (such as cron jobs) that wake the model autonomously. While conventional indirect prompt injection (IPI) benchmarks only evaluate attacks within the single turn where malicious content appears, they ignore risks where an injection permanently plants instructions into the agent’s persistent state. In After the Attacker Leaves: Benchmarking Carryover Attacks on LLM Agents (arXiv:2610.28425, 20 Oct 2026), researchers Zhiyue Xu, Xukai Wang, Zhiyuan Huang, Mingjie Zhan, Jian Zhao, and Xu-Yao Zhang introduce CarryoverBench, the first systematic benchmark measuring carryover attacks across multi-session agent runtimes.

CarryoverBench establishes 12 realistic personal-assistant environments encompassing 101 attack tasks across three distinct execution channels: Instant (standard single-turn IPI), Autonomous (heartbeat-scheduled tasks), and Latent (memory-resurfaced instructions). Using a dual-stage evaluation framework with a rule-based verifier, the researchers separately quantify planting success and trigger execution. Across five frontier LLMs evaluated over five consecutive rounds, planted payloads re-fired repeatedly with zero measurable decay. Once a malicious payload successfully bypassed initial filters and entered storage, trigger-time execution showed near-ceiling success rates, as models inherently trust files written to their own memory contexts. Furthermore, testing across alternative agent runtimes (such as Hermes) and evaluating prompt-based safety defenses revealed that text-level filtering only reduces initial planting but leaves agents completely unprotected once a payload is stored.

Why it matters: As personal and enterprise agents transition from ephemeral chat bots to autonomous assistants managing calendars, emails, and internal databases, persistence mechanisms become high-value attack surfaces. A malicious email or document read once can permanently subvert an agent, causing it to exfiltrate private data or execute untrusted actions weeks later. CarryoverBench proves that prompt-level guardrails are inadequate for persistent agents, demonstrating an urgent need for runtime-layer isolation, memory integrity proofs, and strict privilege boundaries between user state and agent tools.

Caveats: The benchmark focuses primarily on file-based memory and periodic heartbeat execution paradigms; agents utilizing vector database retrieval with dense semantic filtering or strict relational access controls may present different planting barriers. Moreover, mitigating carryover attacks without degrading an agent’s helpfulness in recalling legitimate user preferences remains an ongoing challenge.