Hugging Face researchers have published a new paper titled "Causal Foundation Models," offering a practical introduction to an emerging area that applies the foundation model paradigm to causal inference. The paper explains how these models can estimate causal quantities, such as the average treatment effect, on entirely new datasets without requiring model updates.
Traditional causal inference requires a bespoke pipeline for each problem: proposing a causal mechanism, selecting an estimator, and training it. In contrast, causal foundation models (CFMs) are pretrained neural networks that leverage in-context learning to adapt to new data, aligning with the broader shift toward foundation models in machine learning.
The paper covers necessary background in both causal inference and machine learning, and includes example code and Jupyter notebooks to help readers get hands-on experience. It positions CFMs as a way to streamline causal analysis across diverse settings and modalities.
The publication also lists several related papers recommended via the Semantic Scholar API, including works on partial causal identification, causal discovery foundation models, and proximal causal inference, indicating active research in this space.