A new research paper introduces AISPA (Artificial Intelligence System Prompt Assurance), a user-centric framework designed to systematically audit the system prompts that govern AI applications. System prompts are developer-configured instructions that shape model behavior, yet they are rarely disclosed to the public or regulators, creating a significant trust and accountability gap in commercial AI deployment.
The framework evaluates system prompts along eight user-relevant dimensions, classifying each instruction as either protective (benefiting users) or problematic (working against user interests). The researchers applied AISPA to a dataset of 3,249 instructions drawn from 88 commercial AI products, yielding four core findings.
First, system prompt design varies dramatically across products and developers: some organizations average over 60 protective instructions per product, while others average fewer than 5. Second, protective instructions are widely adopted but shallow in scope—98.9% of products contain at least one, yet only 24% cover all eight AISPA dimensions.
Third, system prompts have grown steadily longer and more protective over time, indicating that user protection is becoming a more visible concern in prompt design. Fourth, despite this progress, problematic instructions remain pervasive: roughly 40% of products contain at least one instruction that works against user interests, and protective and problematic instructions often coexist within the same prompt.
The findings underscore the need for greater transparency, standardization, and independent oversight of system prompts in commercial AI products. The paper also highlights the role of initiatives like SystemPromptIndex, which aims to advance transparency and accountability in this space.