Robot learning has advanced rapidly, enabling manipulation policies to handle more diverse tasks and generalize across environments. Yet reliable execution often hinges on hidden task states that cannot be inferred from a single observation, making interaction history essential. A new paper on Hugging Face addresses this gap by introducing HIDE, a benchmark for evaluating manipulation memory under partial observability.
HIDE comprises 15 tasks spanning repetition counting, historical-state recall, and execution-progress tracking. The benchmark uses randomized initial configurations and includes decision points where similar observations demand different actions depending on what happened earlier.
The authors also propose SEEK, a framework that combines three complementary memory mechanisms to retain historical evidence and track execution state. Evaluations show that existing policies struggle substantially on HIDE, while memory augmentation improves task success in both simulation and real-world experiments.
According to the paper, individual memory mechanisms help on some tasks but can degrade performance on others; their combination achieves the highest average success rate on HIDE among the evaluated configurations. The findings underscore the importance of maintaining internal representations of hidden task states and matching memory design to task-specific information needs.