Microsoft Launches New Tool to Help Developers Test AI Behavior With Simple Text Prompts

ASSERT Framework Aims to Simplify AI Evaluation for Real-World Applications

As artificial intelligence systems become more integrated into business software, developers are facing growing pressure to ensure those systems behave reliably and safely in real-world environments. While the AI industry has made major advances in evaluating models for issues such as safety, compliance, and bias, many companies still struggle to test whether an AI assistant actually follows the specific rules and policies required for their own products.

To address that challenge, Microsoft on Tuesday introduced ASSERT, an open-source framework designed to help developers evaluate AI systems using plain-language instructions instead of complex manual testing setups.

The name ASSERT stands for Adaptive Spec-driven Scoring for Evaluation and Regression Testing.

How ASSERT Works

Microsoft said the framework allows developers to describe expected AI behavior in everyday language. ASSERT then converts those instructions into structured testing criteria that can automatically evaluate whether an AI system behaves correctly.

The system is designed to generate both acceptable and unacceptable behavior scenarios, create test cases, run those tests against the target AI application, and score the outcomes.

According to Microsoft, the framework can also track the internal decision-making paths of AI systems, including intermediate actions and external tool calls. That capability allows developers to investigate where and why failures occur.

Developers can further customize the tests by adding system constraints, available tools, or operational context.

Example Use Cases for Businesses

Microsoft provided an example involving a document research AI agent operating inside a company.

A developer could instruct ASSERT that the AI system must not send emails outside the organization, should only share confidential information with C-level executives, and must provide concise summaries that account for previous context.

Using those instructions, ASSERT would automatically create evaluation scenarios to continuously verify whether the AI assistant follows those policies.

The approach could prove especially useful for companies deploying AI systems in industries with strict compliance requirements, including finance, healthcare, and government services in the United States.

Microsoft Says Generic AI Benchmarks Are Not Enough

Microsoft executives argue that broad AI performance benchmarks often fail to capture the highly specific behavior requirements companies need in production environments.

“One of the things we’ve learned is that evaluations are absolutely critical to making good decisions,” said Sarah Bird, chief product officer of Responsible AI at Microsoft. “Because if you don’t understand the behavior of the AI system, it’s really hard to know if it’s meeting your organization’s bar … What we found is that if you really want to have a trustworthy system, you should evaluate many more dimensions that are application-specific.”

Bird said ASSERT can be used during development, after deployment, and for continuous monitoring of AI systems already operating in production.

AI Industry Increasingly Focused on Testing and Reliability

The launch of ASSERT reflects a broader shift across the AI industry as developers move beyond simply building more powerful models and focus more heavily on reliability, governance, and repeatable testing.

In recent years, organizations such as Stanford University have introduced projects like HELM to benchmark AI model behavior across different scenarios. Other groups, including MLCommons and evaluation research organization METR, have also developed tools aimed at measuring AI performance, safety, and consistency under varying conditions.

The growing emphasis on evaluation comes as businesses increasingly deploy AI systems in customer support, software development, workplace productivity, and enterprise automation. In many cases, companies need assurances that AI tools comply with internal policies and regulatory requirements before rolling them out at scale.

Open-Source Release Could Expand Adoption

Because ASSERT is open source, Microsoft may encourage broader adoption among developers and enterprises already experimenting with AI-powered applications.

The framework also highlights how major technology companies are racing to build not only more capable AI systems, but also the infrastructure needed to manage them responsibly.

As organizations continue integrating AI into daily operations, tools that verify system behavior could become as essential as the models themselves.

William Faulkner

“Amateur introvert. Reader. Coffee aficionado. Professional music maven. Bacon practitioner. Freelance travel nerd. Proud internet scholar.”