Retrieval-based speculative decoding (SD) accelerates language-model generation by drafting tokens copied from existing text, an approach well suited to coding agents that repeatedly reproduce code, logs, and earlier attempts. However, existing retrieval methods struggle inside agent pipelines: much of the reusable text is absent from their corpora or stored in a format that differs from what the agent actually emits, and their fixed draft lengths ignore the fact that acceptable draft length varies across agents and shifts over turns.
To address this, the paper presents AgSpec, a framework that supplies the corpus and draft-length policies that existing retrieval engines lack in coding-agent settings. AgSpec retrieves from three sources — session, workspace, and global corpora — retaining the ongoing session trajectory and indexing opened files in the agent's own emission format.
AgSpec also manages draft length in two stages: it bounds each agent's draft length with an offline-profiled cap, then adapts that length online using feedback from the verification step. This combination targets the variability that fixed-length drafters cannot capture.
On two repository-level multi-agent coding benchmarks, AgSpec outperformed five retrieval-based drafters and EAGLE-3 in most evaluated settings, raising generation throughput over autoregressive decoding by up to 4.37x at batch size 1 and 4.76x at batch size 16. The authors report roughly four times higher throughput than autoregressive decoding overall.
The paper further notes that AgSpec remains effective on benchmarks without a repository or a multi-agent pipeline, suggesting its gains generalize to coding agents more broadly.