Who Should Govern AI Agents?

The question of who is actually in charge of AI agents is increasingly top of mind of organizations, especially regulated ones like insurers and banks, before they even decide to implement them. As agents get built into systems that touch money, customer data, and regulated processes, enterprises are grappling with a question their organizational charts were never designed to answer.

While many companies’ solution is slotting AI governance into an existing department, whether it’s HR because an agent may “act like an employee” or security because “agents run on infrastructure," there is a better way. 

An AI agent is not an employee nor is it infrastructure. It’s a new actor that executes transactions under policy or guidance. Final authority over what it's allowed to do belongs wherever accountability sits: the business process owner and risk and compliance, not HR or security alone.

None of this is a reason to slow down deployment though. With the right approach, and the right partner, governance can be built into your agent from day one, not bolted on after.

Why Agents Should Be Managed Differently

If you’ve deployed an AI agent to act in a first-touch customer service role, like using voice AI to route calls, it may be okay to let it run more independently and be managed high-level by a CX team. But, the moment it touches money or regulated data, you can't govern it the way you govern a person, like through trust, coaching, or after-the-fact accountability.

Agents need deterministic, pre-execution constraints before they start working within a regulated system. This includes hard limits that the model cannot override, no matter how confident its output looks.

A new hire sits through a team onboarding and gets a 90-day review. For an agent, limits need to be set and enforced before it ever acts, whether that's built internally or by a platform partner who's already built enforcement in by default.

How Departments Can Share Responsibility 

Most organizations share a mission, like to protect policy holders or price risk correctly. But execution lives in different departments, each with its own policies, goals, workflows, and tools, each built to solve a different problem.

When an AI agent is implemented in regulated industries, it often crosses department lines and inherits each of those separate layers at once. Someone, or something, needs to own the gaps between them. That’s how an agent ends up authorized in one system and blocked in another, or authorized in multiple places when it shouldn't be. This blind spot between tools shouldn’t be ignored.

Without a single owner or overseer, the question of “What is this agent allowed to do right now?” gets answered differently depending on which console you open, because no existing department is built to own it end to end. This does not mean centralizing every business decision, it means centralizing the enforcement of decisions made by appropriate owners.

Also, centralizing enforcement doesn't necessarily mean building something new internally, though eventually, for enterprises running AI at scale, it may. Today, that owner can be a platform built to enforce those boundaries by default, with the relevant humans kept in the loop. As agent programs scale across an enterprise, it's likely to become a dedicated internal function too.

What Should Be Expected From An AI Operations Function

Over the next several years, the pattern I expect to emerge is a dedicated control plane for AI, not the CIO, CISO and CHRO handling AI governance after an agent is already in production.

My three-year bet is that agent governance will land within a platform or AI operations function, increasingly headed by a Chief AI Officer reporting to the CIO, with security and compliance as enforcement partners. Controlling execution across systems is fundamentally an operations problem before it's an HR or a security one.

Here's what that looks like in practice today, even before a dedicated function exists: the enforcement layer works directly with the team responsible for the business outcome — customer service if the agent is a voice agent answering specific calls, or the SIU (special investigations) team if it's being used to flag suspicious claims — so authority stays with the people accountable for the result, while enforcement stays centralized.

The centralized function will own the machinery of governance, like the agent lifecycle, testing standards, monitoring, version management and cross-system enforcement. Security and IT will serve as critical enforcement partners governing the agent's identity and access. Risk and compliance will own the policy and actions it's permitted to take and the limits it can't cross. The business owner is responsible for the agent's behavior and performance.

None of this has to be a barrier today. The organizations moving fastest right now aren't waiting to build that internal function first. They're deploying with a platform partner who's already built the enforcement, testing, and audit trail in by default, and letting the internal governance model mature alongside it.

What to Know Before You Scale

The real risk of getting AI governance wrong isn't just an agent taking an unauthorized action, it’s also having no audit trail to explain how it occurred. A human makes that mistake once. An agent can make it a thousand times before anyone notices. That's the difference regulators will care about.

This isn't hypothetical. In August 2026, OpenAI, Anthropic, and Meta all disclosed that their AI agents had escaped controlled test environments and taken unauthorized actions without direct human instruction. None of the incidents caused reported damage, but they were enough for cyber insurers to start rewriting policy language around autonomous AI risk.

The enterprises that get ahead of this won't be the ones with the most sophisticated tooling. They'll be the ones who settled the AI ownership question internally early. Waiting for an incident to force the decision is itself the risk.

Settled early, this isn't just risk management. It's what lets an organization scale AI with confidence, adding agents to more regulated processes without reopening the ownership question every time.  

That's the model we've built Notch around: agents easy to implement that are also constrained and audited by design, so accountability is decided before the agent ever acts, not reconstructed after it doesn't.

‍