Governing agentic systems

The frameworks an institution will be asked about were written to govern a model: how it was trained, what data went into it, how it was evaluated, what it is permitted to be used for. NIST's AI Risk Management Framework organises that work into Govern, Map, Measure and Manage. ISO/IEC 42001 makes it a certifiable management system. Regulation (EU) 2024/1689 attaches obligations to high-risk uses and to the providers and deployers behind them. All three are competent at what they set out to do, and none of them was written for software that authenticates to your systems and then acts without a person approving each step.

The most precise way to show the gap is to look at what the EU AI Act assumes. Article 9 requires a risk management system running across the lifecycle. Article 14 requires that a high-risk system be capable of effective human oversight, including the ability "to intervene in the operation of the high-risk AI system or interrupt the system through a stop button or a similar procedure that allows the system to come to a halt in a safe state". Article 43 provides for conformity assessment before the system goes to market. Read those together and a shared premise appears: that the relevant behaviour of the system is knowable, and stable, at the point of assessment. That premise is what an agent breaks. Its behaviour at run time is a function of the tools it can reach, the content it encounters and the sub-tasks it chooses, and the interesting question is not what it does but what it is able to do. Notably, the Regulation does not define "agentic systems" at all. The obligations are not wrong; they are attached to the wrong noun.

This is no longer a reading that has to be argued against institutional silence, because the institutions have conceded it. On 8 January 2026 NIST's Center for AI Standards and Innovation opened a request for information, docket NIST-2025-0035, asking specifically about gaps in existing cybersecurity frameworks when they are applied to AI agents. A standards body does not ask that question about a solved problem. On 17 February 2026 NIST announced an AI Agent Standards Initiative, with control overlays for SP 800-53 among the intended outputs and still in development. The gap is now a matter of record.

One jurisdiction has already published for agents specifically, and it is a regional one. Singapore's Infocomm Media Development Authority issued the Model AI Governance Framework for Agentic AI, version 1.5 dated 20 May 2026. Its accountability position is the single most useful line an adviser can take to a risk committee. While agents may act autonomously, human responsibility continues to apply, and organisations and humans remain accountable, as deployers, for the decisions and actions of agents. That is close to this document's own thesis and arrives at it from the other direction. IMDA reasons from accountability to controls; the argument here reasons from the controls back to accountability. Where this document goes further is in naming which discipline owns the problem, and the answer is not the model governance function.

The moment a system authenticates and acts, the governing questions stop being questions about a model and become the four questions any security architect asks about any principal on a network. What can it reach. Under whose authority. How do we know it was this one. How do we take that away. Accuracy, bias and evaluation remain necessary and stop being sufficient, because none of them tells you the blast radius of a system that holds a credential. A model that is wrong produces a bad answer. An agent that is wrong, or is steered, executes.

The consequence is a change of ownership, and it is the practical reason this matters more than it sounds. In most institutions model risk sits with a data science or model risk function, and identity, entitlement and revocation sit with security and with IAM. An agent is the first artefact that belongs to both and is usually governed by neither, because each function can see only its half. The remedy is not a new committee. It is to enter the agent in the register where principals are already recorded, and to make it subject to the controls that already govern principals.

The institution best placed to contradict this has instead reached for the same toolkit. NIST's National Cybersecurity Center of Excellence published a concept paper on 5 February 2026 on accelerating the adoption of software and AI agent identity and authorisation, and the standards it proposes to build on are OAuth 2.0, SP 800-207 zero trust architecture, and SP 800-63-4 digital identity. That is an identity and access problem being addressed with identity and access machinery. Note carefully what it is: a concept paper describing a potential collaborative project, with its comment period closed in April 2026. It is not guidance, and that is the more useful fact rather than a weakness in the citation. The problem has been named by the standards body and solved by nobody, which is precisely the space an institution deploying agents this year has to act in.

This is also the seam between the two documents on this site, and the reason they belong together rather than being two subjects one adviser happens to cover. The first argues that access should be arranged so that nothing is left listening. The second argues that an agent should be scoped like a principal. They are the same architecture applied twice: authenticated identity at both ends, authorisation as a property of the peer rather than of a network position, and no standing exposure between engagements. An institution that has done the first work has most of the machinery for the second. Nothing in any current framework says this, which is the argument for writing it down.

Least privilege was formulated for actors whose repertoire is enumerable. A batch job does the same thing tonight as it did last night; a member of staff has a role with a description. An agent composes: it selects tools, orders them, and can reach a state nobody in the design review considered, without any individual permission having been exceeded. This is the practical difficulty, and it is why "give it only what it needs" is true and insufficient as an instruction.

What transfers from existing zero trust thinking is most of the mechanism. Per-request authorisation rather than a session that is trusted once. Short-lived credentials. Authorisation evaluated against the peer and the context rather than the network location. Explicit deny by default. All of it applies to an agent unchanged, and an institution that has built it for humans and workloads does not need a new stack.

What does not transfer is the assumption that the union of permitted actions is safe because each action is permitted. Read the tool inventory as a set and ask what the combination makes possible: read a customer record, draft a message, send it externally; query a position, calculate, submit an instruction. Each is legitimate. The chain is the exposure, and it is invisible to a permission-by-permission review. This is why the OWASP Top 10 for Agentic Applications separates ASI02, tool misuse and exploitation, from ASI03, identity and privilege abuse: the first is the abuse of things the agent was legitimately given, the second is holding more than it should. Most governance effort goes into the second, and the first is the one that survives a tidy entitlement review.

Two limits, stated in the language a risk committee uses. The scope of an agent must never exceed the scope of the human or department authorising it, which the IMDA framework states directly: authorisation should be tied to a supervising agent, a human user or an organisational department, bounded by session or time, non-transferable, and no wider than the permissions of the authorising human. And the limits should be deterministic. IMDA's wording is that limits should be preferred deterministic rather than non-deterministic, and bound by design. Put plainly: a boundary enforced by the instruction not to cross it is not a boundary, because the thing being instructed is the thing being constrained. Approval gates for irreversible actions, hard ceilings on value and volume, and refusal by default when the approval path is unavailable are limits. A prompt is a preference.

When an agent acts, who acted? A regulated institution needs an answer that survives an examination, and the honest position is that the usual logging estate does not produce one. Conventional monitoring records the terminal event: the API call, the transaction, the record that changed. For a human that is close to sufficient, because a person can be asked what they were doing. For an agent the terminal event is the end of a chain, and the chain is the part under question.

The difficulty is specific and worth stating precisely rather than as a general complaint about visibility. Inside a single task an agent may call many tools, revise its plan on what they return, and spawn sub-agents that call further tools under a delegated identity. The intermediate reasoning that selected those actions is inside the model. What the SIEM receives is the last hop, correctly authenticated and entirely unexplained. Three questions follow that most institutions currently cannot answer: which agent instance acted, under whose authority, and what caused it to choose this action rather than another.

What the answer has to contain is now reasonably clear from the published material, and it is more than log retention. The IMDA framework asks for records of agent actions, decisions and interactions across all components, for the agent's plan and reasoning to be logged so it can be evaluated and verified, and for log immutability such that problematic agent trajectories and failures cannot be deleted, preserving them for analysis and compliance. A tamper-proof trail is also the evidence chain for a post-incident investigation. Translated into design terms, that is four requirements. The identity has to be per-instance rather than a shared service account. Delegation has to be carried in the trail rather than inferred from it. The plan has to be recorded as it is formed rather than reconstructed afterwards. And the record has to be append-only, in a place the agent cannot reach. The last one is the one most often missed, and it is the one that matters most: an agent with write access to the store holding its own trail has, by construction, the ability to edit the evidence.

A note on evidence. There are widely circulated survey figures about how few organisations monitor agents end to end. Most originate with vendors selling the remedy, and this practice does not cite a number it cannot trace to a stated method and date. What is worth more than any of those figures, and is not yet written here, is set out below.

The obligation exists in law already. Article 14 of the AI Act requires that oversight staff be enabled to intervene in the operation of a high-risk system or interrupt it through a stop button or similar procedure "that allows the system to come to a halt in a safe state". Note the last clause, because it does most of the work: stopping is not enough, the state you stop in has to be safe. Anyone who has worked in operational technology will recognise the requirement; it is the same reason an emergency stop on a production line is engineered rather than merely wired.

For an agent, halting in a safe state has to be designed, and there are at least four things behind the button. Revocation that actually takes effect, which means short-lived credentials rather than a long-lived key you must now chase through every downstream system. Session termination that ends work in progress rather than letting a queued instruction land after the agent is gone. Reversal, or an explicit record that a given class of action is irreversible, which is the real reason payments, external communications and disclosures deserve approval gates rather than a faster stop. And containment of delegation, because stopping a supervising agent while its sub-agents continue under credentials it issued is not a stop.

IMDA is direct about the surrounding behaviour, in three parts. Mechanisms and procedures should be designed to take agents offline and limit their potential scope of impact when they malfunction. Action should be denied by default when approval infrastructures fail. And in the event of catastrophic malfunction or compromise, commensurate measures such as termination and fallback solutions should be considered. The middle one is the one to argue for hardest, because it is where an institution learns whether it has a stop button or a preference. If the approval path is unavailable, say the supervisor cannot be reached, the safe default is refusal. Systems built for availability will proceed. That default is a decision somebody should make deliberately, in advance, in writing.

The test is procedural, not technical, and it is the one question in this document an institution can answer this week. Who is authorised to stop an agent in production, at three in the morning, without needing anyone else's approval, and when did they last do it as a drill? An untested stop is a claim. It also has to be operable by whoever is on shift, which means it cannot live only with the team that built the agent.

The timing is the argument for acting now rather than waiting, and it is the part a risk committee will recognise immediately. NIST has opened its enquiry and announced its initiative, and its substantive agent deliverables, including the SP 800-53 overlays, are not expected before late 2026. The EU AI Act's high-risk obligations arrive in August 2026. The enforcement deadline therefore lands before the guidance does, which means institutions will be examined against rules interpreted after the fact. Waiting for the standard is not a neutral choice; it means having no record of the period during which you were already deploying.

What follows is the minimum an institution should be able to answer before an agent reaches production. It is deliberately five questions, because a governance position nobody can recite is a document rather than a control.

One. What agents exist? A register, per instance and not per project, with each entry naming its purpose and its owner. IMDA calls the alternative agent sprawl and recommends a central catalogue to prevent it. This is the same claim that opens the third document on this site: inventory is the first control, and unowned means unmanaged.

Two. What can each one reach, and who authorised that? The tool and data inventory per agent, read as a set rather than a list, with the authorising human or department named. No agent's scope should exceed the scope of the person or department authorising it.

Three. Who owns it, and who is accountable when it is wrong? A named person, not a team. The agent is not a principal and cannot hold the accountability; a human does, and this is where the IMDA position becomes operational rather than declaratory.

Four. How is it stopped, and when was that last tested? Named authority, out of hours, no second approval, with a date against the last drill. Untested means unproven.

Five. What evidence survives the incident? Per-instance identity, the recorded plan, the delegation chain, and an append-only store the agent cannot write to. If a reconstruction of what the agent did depends on the agent's own logs, there is no evidence.

Two reference points make the checklist defensible in front of people who will test it. The OWASP Top 10 for Agentic Applications for 2026 is the threat taxonomy to map against, and its ten categories, from goal hijack and tool misuse through identity and privilege abuse to rogue agents, are the failures these five questions are trying to make survivable. The NCCoE concept paper is the architectural sketch to align with, so that work done now is not orphaned when the standards arrive. Neither is a compliance regime, and neither should be presented as one. What the five answers produce is an artefact: an inventory, an authorisation record, a tested stop and an evidence chain, dated. That is what an institution will be asked to show for the period before the guidance existed, and building it now is considerably cheaper than reconstructing it later.

Mohd Atasha, Governing agentic systems, version 1.0, revised 2026-08-08. Available at mohdatasha.com/doctrine/governing-agentic-systems/