Diffusion large language models (dLLMs) offer a non-autoregressive alternative to conventional LLMs, but their practical adoption has been held back by inefficient inference. A key culprit, according to a new paper titled Flash-dLLM, is the lack of effective Key-Value (KV) caching and scalable parallel decoding — and the fact that prior acceleration work studied these two mechanisms in isolation, missing the I/O bottlenecks that emerge when they are combined.
The proposed framework, Flash-dLLM, is a training-free inference acceleration system built around two components. Flash-Cache is a fused Triton kernel that merges QKV projection, rotary position embeddings (RoPE), and cache writes into a single operation, while block-scheduled attention handles variable query lengths across a batch. Instead of refreshing the entire cache, it updates only newly decoded tokens and a fixed set of the most-attended decoded tokens, reducing redundant memory movement.
Flash-Verify adds a draft-and-verify decoding strategy in which the dLLM acts as both drafter and verifier, eliminating the need for an auxiliary model. A two-view causal attention mask roughly doubles the number of tokens accepted per decoding step while preserving generation quality.
Experiments on mathematical reasoning and code-generation benchmarks show consistent gains over existing dLLM acceleration methods. On the LLaDA-1.5 model, Flash-dLLM reaches 148–211 tokens per second — 22.3x to 148.2x faster than greedy decoding without caching — and outperforms the prior strongest baseline, Elastic-Cache, by 5.1x on GSM8K and 11.0x on HumanEval. It also uses roughly 48% less GPU memory than Fast-dLLM and scales to batch size 32.
The authors position Flash-dLLM as a step toward making diffusion-based text generation viable for longer sequences and larger batches without retraining, though the reported results are limited to the benchmarks and model configurations tested.