Large language models (LLMs) are transforming AI for games by enabling open-ended, fluid interactive storytelling. However, a new research paper highlights a critical, often overlooked challenge: maintaining long-horizon logical consistency when players can intervene freely.
The study, presented at ICML 2026, formalizes this problem as Narrative Commitment Preservation (NCP). It introduces NCP-Bench, a benchmark built from 100 classic movie synopses, each transformed into an interactive environment with a structured narrative specification—including trajectory, commitments, and initial facts—that can be automatically checked during interaction.
In the benchmark, a narrator agent and a player agent interact for up to 100 turns. The system checks each turn for contradictions, whether the model genuinely responds to the player's actions, and whether key plot points are completed. The authors illustrate the challenge with an example: if the story establishes that “the only key is on the guard,” the player might steal it, break down the door, or lie about having it. The model must respond reasonably while ensuring established facts don't conflict with future turns and that later must-happen plot points are not forgotten.
Evaluation across state-of-the-art LLMs reveals a substantial consistency gap. The best-performing model, GPT-5.2, remains conflict-free in only 42% of cases after 20 turns. Across models, fact conflict rates range from 40% to 68%, and only isolated runs satisfy all achievement commitments within the 100-turn limit. High linguistic quality does not guarantee commitment preservation.
The authors release their data, code, and prompt templates on GitHub, inviting further research into this important area of AI consistency.