General-purpose reward models are becoming a critical bottleneck in scaling robot learning, yet existing methods rely on task-specific anchors like preferences or progress that fail to transfer across different embodiments and data sources. Researchers from Alibaba DAMO Academy introduce RynnValue, an open-source value foundation model that replaces these anchors with temporal distance—the directed cost-to-go from an observation to a language-specified goal.
Because temporal-distance labels can be derived directly from timestamps, RynnValue scales to over 7,000 hours and roughly 3 million instruction-conditioned clips without needing preference or progress annotations. To ensure reliable learning at scale, the model employs random temporal sampling, temporal-order shuffling, and value-isolation attention, which suppress shortcuts that would otherwise make predictions insensitive to failures and regressions.
Trained without preference labels, RynnValue achieves an average Kendall's tau_a of 0.675 on the RBM-EVAL-OOD benchmark, surpassing the fully preference-supervised state of the art (0.655) and more than doubling a progress-only counterpart (0.292). It also generalizes zero-shot to unseen tasks, embodiments, and viewpoints.
When converted into dense rewards via potential-based shaping, RynnValue raises real-world policy success from 52.5% to 72.5% online and from 63.8% to 82.5% offline. These results establish temporal distance as a scalable supervision target and a practical reward interface for generalist robot policies.
The model weights and code are publicly available on GitHub, Hugging Face, and ModelScope.