Long-context modeling is a key capability for large language models, but the quadratic complexity of attention remains a major bottleneck, especially during the compute-intensive prefill phase. FlashPrefill V2, introduced by Hugging Face researchers, evolves the earlier FlashPrefill prototype into a production-ready solution for long-context serving.
The new version introduces a mean correction term that suppresses approximation error, keeping performance degradation manageable even at extreme sparsity levels. It also redesigns the sparse attention operator with PackGQA memory access, warp specialization, and pingpong pipelining, aligning with the latest FlashAttention-3/4 implementations and supporting FP8 inference.
FlashPrefill V2 natively supports paged KV cache and continuous batching, allowing integration as an attention backend in modern inference frameworks such as SGLang. Extensive evaluations on NVIDIA H20 GPUs demonstrate significant speedups over dense baselines.
At 128K context length, FlashPrefill V2 achieves up to 47.26x and 27.19x speedups over FlashAttention-2 under FP8 and BF16 precision, respectively. In FP8, it still delivers a 30.49x speedup against an FA3/4-aligned dense baseline.