Hugging Face has released Kimi K3, a massive open-source AI model with 2.8 trillion parameters, as detailed in a new research paper. The model uses a Mixture-of-Experts (MoE) architecture, activating only 104 billion parameters per token, making it efficient despite its size. It also features native vision capabilities and a context window of 1 million tokens.
Kimi K3 builds on several innovations, including Kimi Delta Attention and Attention Residuals, which enhance information flow across long sequences and deep layers. The Stable LatentMoE mechanism activates 16 out of 896 routed experts per token, contributing to a 2.5x improvement in scaling efficiency over its predecessor, Kimi K2.
Post-training involved reinforcement learning across general, agentic, and coding tasks, with multiple reasoning effort levels. This enables compositional generalization and robust long-horizon execution. The model also benefits from infrastructure advances such as algorithm-system co-design, balanced expert-parallel training, and million-token agentic RL with persistent rollout and sandbox states.
In evaluations, Kimi K3 achieves frontier-level performance on long-horizon coding, agentic, knowledge, reasoning, and vision tasks. While it trails behind proprietary models like Claude Fable 5 and GPT-5.6 Sol, it outperforms other open and proprietary models in its evaluation suite. Hugging Face has released the full model weights to accelerate research and adoption.