Researchers have introduced WING (World Action Learning via INteraction-Centric Spectral Latent Guidance), a framework designed to transfer physical interaction knowledge from egocentric human videos to robot control policies. The work addresses a persistent challenge in robot learning: while large-scale real-world interaction data is essential for general-purpose policies, collecting robot demonstrations remains costly and hard to scale.
Egocentric videos offer an abundant alternative source of human interaction experience with task-relevant semantics. However, direct transfer is difficult for two reasons: latent actions inferred from frame reconstruction can be dominated by nuisance variation such as ego-camera motion, and human and robot behaviors often exhibit different temporal dynamics.
WING tackles these issues in two stages. First, it separates observer-induced motion from hand-object interaction and distills the interaction-centric component into latent actions. Second, it exploits the observation that cross-embodiment task semantics are concentrated in slowly varying temporal structures, identifying shared low-frequency components between egocentric latent actions and robot behaviors in the spectral domain to guide action generation.
The framework reports average success rates of 99.20% on LIBERO, 93.80% on RoboTwin 2.0, and 57.7% on RoboCasa-GR1, and also performs strongly across four real-world manipulation tasks under diverse generalization settings. According to the authors, these results demonstrate that interaction-centric spectral guidance provides an effective and scalable way to transfer physical interaction knowledge from human egocentric video to robot control.