Taking Control of AI Agents

Matt Asay
11 Min Read

Managing permissions for AI agent systems is currently a chaotic mix of vendor-specific settings. We require a standardized framework, akin to ‘Creative Commons,’ for defining agent capabilities.

Unlock potential of AI (artificial intelligence) concept. Hand with golden key and acronym AI. Key to success with use of AI.
Credit: Jirsak / Shutterstock

For roughly two decades, the tech industry was consumed by debates over software licenses. While easily forgotten now, anyone involved in enterprise IT from 2000 to 2020 would recall the endless discussions. We deliberated over licenses like GNU General Public License, Apache License, MIT License, and many others. Vendors faced these issues, but so did enterprises, often scrambling in urgent meetings because a deeply nested dependency contained an unauthorized copyright header. This era saw the rise of entire ecosystems of tools and compliance teams, all focused on a single critical query: What software components are we legitimately allowed to distribute?

At the time, it felt immensely significant. And in many respects, it was. We were establishing the ground rules for how software could be shared, repurposed, and commercialized. Our aim was to transform the wild, unstructured landscape of code into a reliable resource that businesses could integrate into their software supply chains.

Now, history is repeating itself. Only this time, the “code” isn’t a mere library; it’s an autonomous system capable of executing actions on your behalf. This shift explains why the most heated discussions in contemporary AI feel so familiar. Debates rage over open weights versus proprietary models, the origin of training data, and the intricate legalities of liability. The cycle continues.

While these are valid concerns, they might be misdirected.

In the age of agentic AI, a “license” is no longer just a legal document but a technical configuration specifying what the software is authorized to do. Mismanaging these permissions can lead to costly and potentially catastrophic outcomes. Conversely, getting them right becomes tremendously impactful.

The evolving nature of risk

During the open-source movement, the worst consequence of a licensing error was typically legal. Distributing GPL-licensed code within a proprietary product might result in a stern letter, a settlement, and a resolution handled by legal teams.

Agents fundamentally alter the risk landscape. As previously noted, an agent doesn’t just suggest code; it can execute a migration, create a support ticket, modify permissions, dispatch emails, or authorize a refund. Consequently, risk transitions from mere legal liability to tangible, operational consequences. If a large language model generates incorrect text, you get a poorly written paragraph. If an agent hallucinates, it might run a flawed SQL query against a live production database or trigger an excessive cloud resource provisioning event, potentially costing tens of thousands of dollars. These aren’t hypothetical scenarios; they are occurring now, which explains the industry’s sudden preoccupation with safeguards, operational limits, and human oversight mechanisms.

I’ve long argued that for developers, the key AI narrative isn’t about replacement but about management. If AI is the digital intern, then you are the manager. This holds true for code generation and is even more critical for autonomous systems that can act across your entire technology stack. The uncomfortable but undeniable conclusion is that if we’re “employing” synthetic workers, we require organizational equivalents like HR, identity and access management (IAM), and robust internal controls to keep them in check.

The supremacy of the control plane

This paradigm shift clarifies this week’s most significant announcement. When OpenAI introduced Frontier, its primary innovation wasn’t merely improved agents. It was the strategic framing. Frontier is expressly designed to move beyond isolated pilot projects toward enterprise-grade deployment, management, and governance, with integrated permissions and clear boundaries.

Essentially, the AI model is evolving into a modular component. The true differentiator now lies in the sophisticated enterprise control plane that encompasses it.

Media and analyst reactions immediately drew parallels, describing it as “HR for AI coworkers.” Even The Verge highlighted OpenAI’s own comparison to how enterprises manage human teams. Reports on Frontier underscored its emphasis on assigning distinct identities and permissions to agents, rather than allowing them unrestricted operation. Soon after, OpenAI released Lockdown Mode, a security measure designed to mitigate data exfiltration risks from prompt injections by restricting ChatGPT’s interactions with external systems.

These two announcements collectively reveal the industry’s clear trajectory. We’re not merely pursuing “smarter assistants”; we’re striving for governable, permissioned agents that can be securely integrated into core systems of record. This reiterates my consistent viewpoint: the focus is no longer on a model development race, but on a control-plane race.

We are presently navigating the “Wild West” era of agent deployment. While exhilarating, this period is incredibly challenging for enterprises aiming to deploy agents securely and at scale. Developers are currently linking agents using various frameworks, integrating them into business applications, and granting them extensive access, primarily to expedite proof-of-concept demonstrations. The outcome isn’t just tangled code; it’s a labyrinth of logic. This leads to a multitude of semi-autonomous systems exchanging data without any clear audit trail to determine who authorized what action.

This scenario inevitably erodes trust and inflates costs.

I refer to this as the “AI trust tax.” Each time an AI system errs and requires human intervention to rectify it, the actual operational cost of that system rises. The only way to mitigate this tax is to shift from treating governance as a mere policy issue to an architectural imperative. This means implementing the principle of least privilege for agents, just as we do for humans. It entails distinct separation between “draft” and “send” actions. It requires making “read-only” a fundamental capability, not an afterthought. It demands auditable action logs and workflows that can be reversed. Crucially, it means designing your agent system with the understanding that it will be subjected to attacks.

This is also why I’ve consistently emphasized that agent memory poses a database challenge. If agent memory functions as a state store, it requires the identical protections afforded to any state store: firewalls, audit trails, and access privileges. Agentic permissions are a logical extension of this principle. Once an agent is empowered to act, it necessitates privilege boundaries as stringent as those applied to any database administrator.

Permissions: The new paradigm of collaboration

In the early 2000s, open-source licenses achieved a remarkable feat: they streamlined software reuse through standardization and widespread comprehension. Licenses like Apache and MIT minimized legal ambiguity, while the GPL leveraged legal frameworks to promote a culture of sharing.

Now, we need a similar framework tailored for AI agents.

Currently, permissions are a fragmented mess of vendor-specific configurations. Each platform implements its own method for defining action scopes. One might integrate an approval workflow, while another delegates the issue to your existing identity and access management team (often a challenging prospect!). This lack of standardization will impede adoption rather than accelerate it. Enterprises cannot effectively scale agent deployments until they can define straightforward rules. We must be able to specify that an agent can access production data but not modify it. We need to enable an agent to draft emails without the authority to send them. We must mandate that an agent can provision infrastructure only within a sandboxed environment, with defined quotas, or that it requires human consent before executing any destructive action.

We need a system akin to “Creative Commons” for agent conduct: a standardized lexicon for agentic scopes that can be universally understood and applied across different platforms.

The open-source movement eventually established two crucial elements that reassured enterprises: standardized licenses and the tools to meticulously inventory their software components. A Software Bill of Materials (SBOM) serves as the modern incarnation of this inventory, and standards like System Package Data Exchange (SPDX) are vital for ensuring interoperable licensing and supply-chain transparency. The agent landscape requires an analogous progression. We need a machine-readable declaration outlining an agent’s identity and its authorized capabilities. Whether you name it permissions.yaml is secondary; its portability is paramount.

So, indeed, a new form of “open source” is necessary, but not the traditional kind. We’re not talking about the familiar open-source community haggling over the availability of open-source AI without its training data, or similar debates. Those are concerns for yesterday’s open-source program offices, largely irrelevant to the current trajectory of software development.

No, the future revolves around agents and the permissions that govern their actions. Back in 2014, I suggested we were already operating in a “post–open source world.” When I wrote that “software matters more than ever, but its licensing matters less and less,” I certainly wasn’t envisioning agents. I still believe that statement holds true, though perhaps there’s renewed space for “licensing” discussions as we strive to enable agents to safely and scalably interact.

Artificial IntelligenceGenerative AIOpen SourceSoftware DevelopmentIdentity and Access ManagementSecurity
Share This Article
Leave a Comment

Leave a Reply

Your email address will not be published. Required fields are marked *