AI AGENT GOVERNANCE
Govern the agent, not just the prompt.
A prompt turns input into output. An agent thinks, calls tools, reads results, and answers. Mowa judges the whole trajectory: did it call the right tools, avoid the wrong ones, and ground its answer in what came back.
THE SHIFT
From output to trajectory.
An agent is a prompt plus tools. The unit you evaluate is no longer a single answer, it is the path: every tool call, its arguments, the result, and the final response. Same versioned registry, same contracts, same attribution, now over the full run.
- An agent is a prompt version with a tool set attached
- Runs record the trajectory: calls, arguments, results, answer
- Everything you built for prompts carries over to agents
1 think classify the request
2 call lookup(customer_id)
3 result plan: pro, status: active
4 answer grounded in the result
TOOLS
Any tool, and a mock for the ones you cannot call.
Tools are anything an agent uses: Figma, a search API, an internal service. Define one with a name and a JSON schema. Every tool ships a mock, so agents run in eval without touching real systems, and internal tools are always mocked so nothing dangerous fires.
- Define any tool with typed, JSON-schema arguments
- Mock strategies: static, templated, dataset-backed, or model-simulated
- Internal tools are always mocked, safe by construction
🔎 web-search external · static mock
🗂️ customer-lookup internal · simulated
💸 issue-refund internal · never real
ATTRIBUTION
Now prompt, model, or tools.
When an agent regresses, isolate whether it came from your prompt, the model you swapped, or a tool that changed. Agents add a third axis to the same causal decomposition.
- Prompt versus model versus tool-set isolation
- Every trajectory stored as an immutable record
- Production tool calls captured through the gateway
prompt 0 model -2 tools -9 cause: tool change
CONTRACTS
Assert what it must call, and must never call.
The contract now reads the trajectory. Assert that the agent called the right tool with the right arguments, in a sane order, and never touched the tools it should not. Deterministic matchers first, a model judge over the whole path second.
- called(tool, args) and never-called(tool) as deterministic checks
- Order and argument-schema validation on every call
- A model judge over the trajectory for the fuzzy calls
must call lookup before refund
must never refund without approval
args valid order_id present
verdict pass
How it runs
one loop, every record kept
01 prompts
versioned, reviewed
02 datasets
rows + synthetic
03 runs
versions × models
04 scorers
your judgment
05 experiments
pass rates
AI AGENT GOVERNANCE
Ship agents you can defend.
Define the tools, write the contract, and let every release prove the agent calls what it should and nothing it should not.