Latent world models are a cornerstone of modern reinforcement learning, enabling agents to predict future states in a compact representation space. However, their effectiveness hinges on the quality of the learned latent distribution. A common approach, used in the LeWorldModel (LeWM) framework, regularizes latents toward an isotropic Gaussian using the Epps-Pulley (EP) objective.
In a new paper, researchers identify a critical flaw in the EP objective: its corrective gradients vanish rapidly for isolated tail samples, leaving heavy-tailed deviations under-controlled. To address this, they introduce QQWorld, a quantile-quantile matching objective that directly aligns projected latent samples with rank-matched Gaussian quantiles. This ensures effective corrective gradients even in the tails.
The team also develops cross-batch QQ, which enlarges the effective ranking pool by using detached samples from previous batches, and they characterize its bias-variance trade-off. Across four control environments, QQWorld improves the average planning success rate of LeWM while consistently yielding better Gaussian alignment and thinner latent tails.
The work offers a simple yet powerful modification to latent world model training, with potential implications for sample-efficient planning in reinforcement learning.