Video world models have made remarkable progress in generating realistic and dynamic scenes, but they have largely ignored a key aspect of user engagement: social interaction with the characters within those worlds. To address this, researchers from Hugging Face have introduced HelloWorld, a novel video world model that allows users to interact with on-screen characters in a socially meaningful way.
With a single button press, users can prompt the character to respond directly to the camera—turning to face the viewer, waving, nodding, or even speaking a short greeting. This capability bridges the gap between passive video generation and interactive experiences, opening new possibilities for gaming, virtual reality, and digital storytelling.
The core innovation lies in a self-distillation pipeline that fine-tunes the video generation model on data it synthesizes itself. Each generated clip includes both social interactions and camera motion, enabling the model to learn camera-pose conditioning without degrading interaction quality. This approach ensures that the character's responses feel natural and contextually appropriate.
At inference, HelloWorld introduces a training-free module that determines when the interaction occurs. Upon a button press, this module modulates the cross-attention masks of the Diffusion Transformer (DiT), ensuring that the interaction-related text prompt attends only to the frames within the press window. This temporally localizes the character's response, making the interaction feel immediate and responsive.
To evaluate the model, the team built HelloWorldBench, a 400-sample benchmark that includes three social interaction metrics alongside three conventional metrics. Experiments show that HelloWorld surpasses a variety of baselines in interaction quality while maintaining state-of-the-art picture aesthetics and camera-pose following. The project is open-source and available on GitHub.