DeepSeek-V4.1-Flash, a new multimodal Mixture-of-Experts (MoE) model, targets the compute, storage, and bandwidth bottlenecks that limit the deployment cost of long-context agentic workloads. According to the paper, the model carries 552B backbone parameters and supports contexts of up to one million tokens.
The model uses a Causal Encoder-Decoder (CED) architecture that activates 16B parameters per token during decode but only 8B parameters during prefill, a split the authors say substantially improves cost efficiency for agentic workloads, where prefill has remained computationally expensive.
To push KV cache compression further, DeepSeek-V4.1-Flash combines cross-layer KV cache reuse in Compressed Sparse Attention 2 (CSA2) with FP4 KV caching. These designs reduce the global KV cache footprint — always held in HBM — to 890 bytes per token, roughly one quarter of the corresponding footprint of DeepSeek-V4-Flash.
A deployment optimization called SWA Bounded Replay further reduces the persistent KV cache footprint, which resides on SSD or in host memory, to about one eighth of DeepSeek-V4-Flash's. The paper reports that the model nevertheless delivers substantially better performance than the baseline.
The authors also describe a streamlined DeepSeek-V4 architecture with several efficient extensions. DeepSeek-V4.1-Flash was pretrained on a multimodal corpus of 45T tokens and then post-trained, yielding strong results across text-based and multimodal agentic scenarios. Model checkpoints are available on Hugging Face.