Researchers have introduced PDE-JEPA, a framework for predictive representation learning of parametric partial differential equations (PDEs), according to a paper posted on Hugging Face. The work argues that physical trajectories carry more than snapshots of a system: they also encode how states evolve under governing conditions, yet representation learning for parametric PDEs has largely relied on reconstruction-based objectives that focus on recovering observed physical fields.
The paper investigates predictive representation pretraining as an alternative to reconstruction-based learning. The authors report that predictive representations preserve rich physical information, but note that this advantage alone does not guarantee accurate field evolution. To address this gap, PDE-JEPA first trains an encoder using masked-latent prediction to capture underlying regularities of PDE dynamics.
The framework then adds a geometry projector that aligns latent trajectory geometry with the evolution geometry of physical fields, explicitly adapting the pretrained representation toward a more dynamics-aligned state space. Building on this geometry-aligned latent space, the authors develop a physics-structured latent predictor that decomposes the dynamics into parameter-independent evolution and parameter-dependent response components.
Extensive experiments on nine widely used PDE benchmarks show the framework outperforming existing state-of-the-art methods by an average of 33.4% in-distribution and achieving an average improvement of 51.4% when extrapolating to unseen governing parameters. The authors state that, to their knowledge, this is the first systematic investigation of JEPA for parametric PDEs and the first to make it work effectively for PDE forecasting.
The paper concludes that informative JEPA representations alone do not guarantee accurate rollout, and positions PDE-JEPA as a way to bridge that gap through physics-aligned latent geometry and structured latent prediction. A project page is available at the link provided in the paper.