Autonomous agents have advanced quickly in general-purpose computer use, but the paper argues that reliable automation of professional industrial engineering remains out of reach. Engineering workflows require reasoning over geometric and physical constraints and dependencies that must be preserved across software and design stages.
To measure progress, the researchers present EngiWorld, described as the first benchmark structured around the complete design loop. It contains 1,301 expert-curated tasks spanning 6 engineering domains — CAD, CAE, CAM, BIM, EDA, and 3D visualization — and 26 professional software platforms, with both GUI and CLI interfaces and six task types ranging from software selection to open-ended tasks.
The paper also introduces an artifact-centric evaluation methodology built on a unified domain-verifier suite. This suite programmatically checks the geometric validity, physical feasibility, and rule compliance of final and intermediate artifacts, and scores quantitative design tasks continuously by specification attainment rather than binary success.
Evaluating seven frontier models reveals a substantial capability gap: the strongest model achieves an EngiScore of only 44.3, and just 3.6% of multi-software attempts succeed. The authors conclude that reliable end-to-end automation of professional engineering workflows remains out of reach, and position EngiWorld as a rigorous foundation for measuring future progress.