Long-context LLM agents accumulate interaction histories that strain KV-cache memory and attention computation. While sparse attention can reduce these costs, heterogeneous cache representations and workflows make it difficult to integrate with existing inference engines, and prior sparse-serving abstractions support only specific layouts or workflows.
Researchers present SparseEngine, a ground-up, sparse-first inference engine. Its shared lifecycle contract allows each sparse method to control its KV representation and computation while coordinating state transitions with common serving infrastructure. SparseEngine supports 15 methods across four categories.
The engine introduces Chain Cache for cross-request state management, which resumes KV-eviction methods from retained history, and controllable Prefix-Cache Pruning, which removes KV from selected history regions while preserving logical-prefix matching.
According to the paper, SparseEngine maintains method quality while delivering over 10x higher throughput with KV eviction, over 2.5x faster decoding at matched concurrency than vLLM, and over 2x end-to-end speedup on agent benchmarks. The code is available at https://github.com/CURRENTF/SparseEngine.