How do we Govern AI that Thinks and Acts for Itself? That Really is the Million-Dollar Question.

How do we govern AI that thinks and acts for itself? That really is the million-dollar question

Unlike the AI we’re now really used to, like chatbots or recommendation engines, agentic AI goes so much further than just responding to our requests and generating a reply, it acts. (Cool, huh!) These autonomous systems can set their own goals, make decisions, and carry out actions independently, with limited or no human input.

For Chief Data Officers, I think this additional step on the AI evolution ladder is both a massive opportunity and a huge messy pain to the governance challenges you’re already dealing with.

Agentic AI stands apart from classical machine learning. Rather than producing simple outputs from predictive inputs, it links multiple AI models in a chain. This is where the output of one feeds directly into another. This autonomy underpins four defining traits, let’s walk through them.

Underspecification.

AI is given a broad objective but no clear path to follow.  I could ask it to “optimise marketing campaign performance.” However, I’ve not told how to achieve this. I’ve provided no specific channels, constraints, or ROI benchmarks are provided. The AI autonomously decides to reallocate budgets, halt low-performing campaigns, and even generate new content.

Long-term Planning.

It builds upon its own decisions over time. A customer support AI agent is tasked with reducing average resolution time, for example. Over time, it identifies patterns in user issues, rewrites FAQ entries, reroutes tickets based on historical resolution success, and starts proactively emailing customers with solutions before they ask. (Gosh, sounds far too good to be true, doesn’t it?)

Goal-directedness.

It takes initiative to reach outcomes, not just respond to queries. An AI agent responsible for sales enablement notices that certain phrases increase conversion in email outreach. It then begins tailoring and optimising email campaigns with increasingly persuasive language. We’ll come onto the risk of all of these further in a moment, but this one, I think we can see how slippery a slope we could fall down quickly without correct governance in place.

Directedness of Impact.

Some systems operate entirely without human oversight. For example, an autonomous procurement agent is connected to supplier portals and payment systems. It identifies a potential cost saving and negotiates a switch to a lower-cost vendor—initiating contract changes and updating logistics settings without human sign-off.

This shift from reaction to action introduces a fundamental truth: greater autonomy equals greater risk. And for data leaders, that should set off alarm bells, flashing lights and keep you up at night (okay, not really that last part, but you get the idea).

The risk landscape is undoubtedly growing. The emergence of agentic AI dramatically amplifies existing generative AI risks. Misinformation, errant decision-making, and security vulnerabilities are chief among them. But it also introduces new risks, as these systems operate in less supervised, often less interpretable ways.

With fewer humans in the loop, fewer domain experts to provide context or course-correct, CDOs must address a critical question: how do we govern AI that thinks and acts for itself? That really is the million dollar question.

Governance Must Be Multi-Layered

Effective governance of agentic AI is not a bolt-on, it absolutely has to be foundational. A comprehensive approach spans technical, procedural, and organisational safeguards. Through my research, here are some of the areas and questions we should be asking.

From a technical perspective, can AI requests or systems be paused or shut down? At which points must AI await human approval? Are PII and other sensitive data adequately masked or sanitised?

There has to be adequate process controls. CDO’s must establish risk-based permissions, defining which actions AI must never take autonomously. There has to be a level of auditability, is it possible to trace back how a decision was made? And there must be a plan for ongoing evaluation is essential for compliance, safety, and performance.

From an organisational accountability perspective there has to be clear ownership. When all metaphorical poop hit’s the proverbial fan, who is responsible when things go wrong? Are your AI use cases aligned with existing legal and ethical standards? Are your AI suppliers accountable for model behaviour?

Guard Rails Must Be Built into the Architecture

To deploy agentic AI responsibly, controls must be embedded across the technology stack. Let’s look at what those gaurdrails could be with some examples from across our industry.

Model Layer

Protect against prompts or behaviours that violate organisational policy or ethical standards. For example, without guardrails at the model layer, a customer could use your chatbot could prompt it to give financial advice, violating regulatory compliance. Therefore, at the model layer you should include some form of safety filter that blocks certain topics (e.g., medical, legal, or financial advice), and screens outputs for offensive or discriminatory language using toxicity classifiers.

Orchestration Layer.

Detect infinite loops or runaway processes that risk costly system failures. One of the most common uses of Agentic AI across all enterprises will be dealing with internal tickets. However, what if due to a misconfiguration, it keeps triggering itself by interpreting its own confirmation message as a new request, consuming compute resources and overwhelming system logs? You need include loop detection logic, maximum recursion limits, and timeout thresholds.

Tool Layer.

Limit each tool to approved use cases using role-based access control. Let’s go back and use a marketing example. An AI marketing assistant is given tools for generating campaign copy and uploading it to a content management system. Seems simple enough. However, without proper restrictions, it might gain access to payroll or HR data APIs because of shared credentials. Eek. Not so simple, and certainly not good. Therefore you need some king of role-based access control (RBAC), you define that this AI can only interact with CMS tools—not ERP, CRM, or HR systems. Each agent is assigned permissions aligned to its scope of work. Creating a nice little ring fence of safety.

System Testing.

Employ red-teaming exercises to uncover weaknesses before deployment. Before launching a customer-facing AI, your team runs red-teaming simulations, including prompt injection attacks, misuse scenarios, and attempts to trick the model into leaking confidential data. Basically, give it a really good tickle and see if it confesses any company secrets. These simulations expose edge cases, ethical blind spots, and security weaknesses that are then used to update prompt filters, refine model fine-tuning, or change agent logic before release.

Ongoing Evaluation.

Use observability tools to detect hallucinations, policy breaches, or degraded performance in real time. You can’t just let these systems go into the wild and never check on them again. Let’s use sales as an example again. Post-deployment, your AI sales assistant starts to drift in performance—responding inaccurately to product queries, really rather unhelpful. Your observability stack detects a spike in hallucination rates and policy violations using metrics dashboards, log analysis, and feedback loops from users. You’ve integrated tools like model traceability logs, real-time compliance checkers, and anomaly detection systems that flag problematic behaviour before it escalates. It’s inevitable that some models will go slightly off track, but by having ongoing evaluation you limit the risk of them doing real damage at scale.

Agentic AI is not speculative, it is well and truly here. For CDOs, the challenge is not whether to engage, but how to govern. This is not solely about risk mitigation. It’s about control, accountability, and trust.

Before permitting AI to act on behalf of your business, ask yourself, do we have the right guard rails in place?

We’re in the age of agentic AI and the responsibility for ethical, safe, and effective use doesn’t lie with the machine—it lies with us.

Author

Want to talk more about Agentic AI? Come join us at the Agentic Webinar, where we’re discussing how it’s going to impact your workforce planning