Multi-agent LLM systems increasingly pair heterogeneous models so that each agent can specialize. But when agents communicate in text, every receiver must re-process context the sender has already computed. Reusing the sender's key-value (KV) cache removes that redundancy — yet doing so across model families is difficult because of mismatched tokenizers, model depth, and KV representations.
A new paper, Prefill-Free Cross-Family KV Cache Transfer for Heterogeneous Multi-Agent LLMs, proposes HeteroFold, a method that performs prefill-free cross-family KV cache transfer while keeping both the sender and the receiver frozen. According to the authors, HeteroFold aligns model structures, maps the sender's cache into the receiver's representation space, and calibrates the transferred cache to preserve the receiver's behavior.
The work targets models including Llama, Qwen, and Ministral. HeteroFold aligns tokens across differing tokenizers and layers across differing architectures, then calibrates the transferred cache so the receiver's attention patterns and outputs are preserved. Because the learned mappings are used instead of retraining either model, the receiver can decode directly from the transferred cache without processing the original context again.
Across six transfer directions, the authors report that HeteroFold achieves the best cache-transfer performance on all four long-context benchmarks and on most short-context settings, and that it matches text-based communication on the multi-agent benchmark. At a 32K context length, they report that a Llama-3.1-8B to Ministral-3-14B transfer is about 10.7× faster than Native Prefill and 1.18–1.47× faster than the prefill-free baselines Dense Latent and KV Ridge.
The authors frame the result as a step toward making KV caches reusable across model-family boundaries, so that heterogeneous LLMs can reuse one another's computation instead of repeatedly recomputing the same context.