Tool-calling agents are becoming a cornerstone of enterprise AI, but training and evaluating them at scale is held back by business and legal restrictions on real enterprise systems, data, and database schemas. A new research paper posted to Hugging Face proposes a way around that bottleneck.
The paper introduces Synthesis Through Simulation (STS), described as a schema-free data synthesis paradigm. In STS, an LLM agent generates data by executing operations against policy-enforcing APIs inside simulated enterprise environments. Because the data is produced through the same environment that defines what is valid, the authors say STS guarantees structural validity by construction and separates validity enforcement from distribution modeling.
The framework's domain-agnostic agent, called the Generalist Populator (GP), targets the remaining challenges of distributional fidelity and synthesis scalability. According to the paper, GP reaches 0.88 average marginal fidelity and 100% constraint satisfaction across all ten environments without access to database schemas. By contrast, the authors report that statistical synthesizers are inapplicable to seven of the environments due to necessary seed-data requirements, while schema-privileged agents fail 82% of trajectories on the airline environment's tightly coupled workflows because of brittle task composition.
The full framework, all ten environments, and the generated datasets are released as open source at github.com/SAP/synthesis-through-simulation.