Nothing on the shelf tells you how to govern an autonomous agent running live in your business. ISO 42001 exists and the EU AI Act is arriving, and both are worth reading, but they were written for AI systems: something that produces an output a person then uses. An agent is a different animal. It acts on its own, it chains one action into the next, and it holds credentials to your real systems. The gap between those two things is where the risk lives.

This began as a talk for the CyberEdBoard, "Governing in the Void", on how a CISO builds AI security controls before the standards catch up. Here it is in full.

It is not theory. We run a fleet of named agents in production across security operations, vulnerability management, threat modelling, procurement, sales and the executive office. What follows is what we actually enforce, and what we would set up on day one if we were starting again today.

The constitutional principles

Five rules govern every agent in our fleets. They are deliberately short enough to memorise and strict enough to audit.

The constitution
Five rules, each one testable
AI proposes; guardrails decide
Agents investigate, reason and draft. Humans commit, approve and release. No exceptions for spend, production changes or external communication.
SpendProductionExternal comms
Confidence-threshold escalation over guessing
Below the threshold, the agent escalates with its work shown. It never guesses its way past uncertainty.
Explicit thresholdEscalation default
Evidence-gated auto-action
An agent may act autonomously only when the evidence chain supports the action. 'It seemed right' is not an evidence chain.
InputsReasoningConfidence
Adversarial testing before production trust
Every agent gets attacked before it earns autonomy: prompt injection in its inputs, poisoned context, contradictory evidence, malformed data.
Prompt injectionPoisoned context
Named accountability
Penny Pryce drafts procurement; a human owns procurement. Every agent class has a named human accountable for its decisions.
One agentOne owner
Short enough that an engineer can recite them in a design review, specific enough that an auditor can ask for the evidence. If a rule cannot be tested, it is a value, not a control.

The three things most AI governance programmes leave out

The three
Identity, guardrails and audit - in that order
Agent identity
An agent logged into Okta, GitHub, AWS and Salesforce is a privileged account that behaves nothing like a person: no working hours, no typing speed, no habits your monitoring was tuned for. Treat it as the privileged account it is: dedicated credentials, least privilege scoped per integration, full session logging, and a periodic review of whether it still needs what it has. When we started granting people admin rights only for the task in hand, agents got the same treatment.
Dedicated credentialsPer-integration scopeRecertification
Guardrail architecture
Guardrails belong at the layer where action happens, not bolted onto the model. Our agents never hold standing write access to production: they draft PRs humans merge, draft emails humans send, prepare approval packs humans sign. The guardrail is not a content filter on the model. It is the question of who is allowed to make a change take effect, and the answer is always a person.
No standing write accessDraft then commit
Audit by construction
Every agent decision emits its evidence chain - inputs, reasoning, confidence, action or escalation. Six months of autonomous SOC with zero false positives is only credible because every one of those decisions is inspectable after the fact. Regulators will ask. Build it before they do.
Inspectable decisionsRetained evidence
Most AI governance programmes start at the model and work outwards. Start instead at the account the agent logs in with, and at the moment its work takes effect. That is where an agent can actually do damage.

Mapping to ISO 42001 and the EU AI Act

Here is the good news: an engineering-led agent governance programme satisfies most of what is coming.

| Framework | What it asks for | What satisfies it | |---|---|---| | ISO 42001 (AI management systems) | Documented AI policy, risk assessment, roles, controls | Principles 1-5 are that policy, enforced in code rather than laminated on a wall | | EU AI Act | Risk classification, transparency, human oversight, logging | Human-in-the-loop by design plus audit-by-construction covers the operational requirements for the agent tier most enterprises run | | Gartner AI TRiSM | Trust, risk and security management for AI | Maps almost one-to-one onto the three things above |

The accreditation trap to avoid: treating AI governance as a documentation exercise. We run ISO 27001 and ISO 42001 tracks in parallel with an engineering-led approach precisely because the checkbox version of AI governance protects nobody - not your customers, not your board, and not you when an agent misbehaves in production.

Start smaller than you think

The organisations that fail at agentic AI either never start, or start with mission-critical autonomy. The winning path is a portfolio of boring, bounded, high-volume decisions with humans explicitly in the loop, then more autonomy as evidence accumulates.

Sequencing
Autonomy is a ladder, and every rung is earned
Bounded verdicts, humans action everythingWeeks, not months
Phishing verdicts, vulnerability re-classification, a renewal radar. High volume, mostly benign, crisp right answer. The agent reasons; a human does every action.
Draft and prepare, humans commitOnce verdicts hold up
The agent drafts the PR, the email, the approval pack. Nothing reaches production or a customer without a human pressing the button.
Automatic action where a mistake is easy to undoOnce the evidence record exists
Only actions you could reverse in a minute and would see in a log: remove a confirmed phishing message, sign a session out. Gated on evidence and confidence, recorded either way.
Wider autonomy, same constitutionOnly where it is proven
Scope grows where the decision record supports it. The rules never relax; only the range of decisions they cover does.
Autonomy granted
Each rung needs a body of evidence from the rung below it: decisions logged, verdicts checked against reality, adversarial tests passed. Nobody skips to the top, and an agent that fails a rung goes back one.

Within a year you will have what we have: a fleet of named colleagues who never sleep, never guess, and always show their work.

Do this next
A governance programme you can start this month
1Write your five rules and get them signed off by the person who owns the risk. Five testable sentences beat a forty-page policy nobody reads.
2List every account your AI already logs in with, then cut each one back to only what that integration needs. Most organisations find shared credentials and permanent write access at this step.
3Move the guardrail to the commit path: remove standing production write access from agents and make the human merge, send or sign the only route to effect.
4Make every agent decision emit inputs, reasoning, confidence and outcome to a store you can query in a year. Decide the retention period now, before a regulator picks it for you.
5Run an adversarial test before granting autonomy - prompt injection in real inputs, poisoned context, contradictory evidence - and keep the results as the evidence that autonomy was earned.
6Map the five rules onto ISO 42001 clauses and EU AI Act obligations once, in a table. That table is most of your audit answer, and it takes an afternoon.
Ask these
Questions that expose whether your AI governance is real
Can we list every agent or AI integration in production, with the named human accountable for each one?
What is the most consequential thing our AI can do today without a human pressing a button - and did anyone decide that deliberately?
If a regulator asked us to justify one specific automated decision from three months ago, could we produce the evidence chain?
Which agent credentials hold standing write access, and to what?
Has any agent been adversarially tested, and do we still have the results?
Is our AI policy enforced anywhere in code, or does it only exist as a document?

Governance questions are usually the blocker, not the engineering. If that is where you are stuck, bring it to a session - it is the single most common thing we are asked about.