Researchers have introduced MaP-WAM, a framework that reframes how robotic policies handle long-horizon memory. According to the paper, most mainstream robotic policies adopt a Markovian formulation, yet many real-world manipulation tasks are inherently non-Markovian and demand memory that extends well beyond the current observation.
Existing memory mechanisms — language summaries, growing visual windows, or combinations of both — can lose fine-grained visual evidence or force a trade-off between history coverage and execution efficiency. MaP-WAM addresses this by decomposing memory-dependent world-action modeling into two parts: memory-grounded planning and plan-conditioned execution.
The framework represents memory as completed segment records containing language instructions and sparse visual context, then converts that episodic memory into compact plans made up of a next segment-level language plan and corresponding visual guidance. A World-Action-Progress (WAP) model executes each plan over an unknown duration by jointly predicting action chunks and their execution progress at inference time, calibrating predicted progress through plan-observation alignment to enable adaptive segment transitions and closed-loop context updates.
Because the executor context length stays fixed, structured attention allows key-value caching in both planning and execution. The paper reports state-of-the-art results on RMBench with an 83.3% success rate and 78.0% success on real-robot tasks, while executor inference latency remains approximately constant as task history grows.
Project page and code are available via the links provided by the authors.