CONTROL BEFORE AUTONOMY
How should a business AI agent be governed?
Govern a business AI agent as a versioned operating system, not as a chatbot. Name an accountable owner, limit its identity and tools, require approval for consequential effects, record every external action, define stop and rollback authority, and reassess the exact model, prompt, data, permissions, and policy on a fixed date. Governance must follow the workflow risk and actual autonomy.
Key takeaways
How should a business AI agent be governed?
- 01
One named owner remains accountable for the operating outcome.
- 02
Permissions should be narrower than the agent's technical capability.
- 03
Production remains reversible and tied to one evaluated version.
Minimum control layers
| Layer | Question | Required evidence | Owner |
|---|---|---|---|
| Mandate | What may the agent do? | Scope and prohibited effects | Business owner |
| Identity | Which data and systems may it reach? | Accounts and permission map | System owner |
| Decision | Which effects need approval? | Approval and veto log | Approver |
| Operations | Can the effect be contained and reversed? | Incident and rollback test | Operator |
Govern the effect, not only the answer
The key boundary is what the system can change. Drafting a price, committing a price, sending a price, and recording a contract are different effects. Each needs a separate permission, validation rule, and trace.
Human approval must show the actual recipient, amount, object, and intended change. A generic confirmation button is not a meaningful control.
Reassess after every material change
A new model, prompt, retrieval source, connector, memory policy, permission, or business rule can invalidate prior evidence. Keep the operating configuration versioned and return to the appropriate evaluation stage when it changes.
WORKED EXAMPLE
A five-step rollback
Disable writes, revoke the agent identity, preserve the trace, restore the last known safe state, and route work to the manual process. The rollback is only credible if it has been tested before production and can be executed by a named person.
- Named stop authority
- Versioned permissions
- Tested manual fallback
Sources and limits
Sources and limits
These sources bound the answer. They do not turn one published case into a promise for your organization.
- 01NIST AI Risk Management Framework ↗
A lifecycle framework for governing and measuring AI risk.
- 02EU AI Act official portal ↗
Current regulatory orientation and implementation material.
- 03MITRE ATLAS ↗
Threat, technique, and mitigation knowledge for AI systems.