Governing Non-Human Identities in Agentic Systems

The security conversation around AI is shifting from model outputs to operational behavior. Organizations are deploying autonomous agents that can invoke tools, modify infrastructure, and participate directly in production workflows.

In practice, a new and particularly risky class of non-human identities (NHIs) is emerging. Unlike static service accounts, these systems exhibit non-deterministic behavior driven by language models and tool access. They operate continuously and often inherit broad privileges to complete tasks.

The scale of adoption is increasingly visible in operational risk data. Reports indicate that 87% of enterprises already have Microsoft Copilot enabled, while more than half of AI agents access sensitive information, and 90% operate with permissions broader than required. These agents can hold up to ten times the privileges they need and move data at far higher volumes than human users. Complementary surveys report that 80% of organizations have observed unintended agent actions, including unauthorized access to systems or data.

The behavior of an autonomous agent is hard to assess with traditional evaluation methods. Actions that appear correct can hide malicious logic, and the impact of errors or attacks goes beyond the typical risks associated with working with language models.

This raises an important question: how can we maintain architectural control over an entity whose actions look legitimate in logs and at the API level, yet still result in a security breach? Addressing this challenge requires understanding emerging attacker tactics and rethinking traditional security approaches.

 

How AI Agents Are Used in Real Systems

AI agents are already being deployed across real operational environments. The examples below show how they are used across security and DevOps domains.

Traditional SOC approaches struggle to keep up with the volume of telemetry and the speed of modern attacks. Most monitoring centers already use some level of automation, yet the critical stages of analysis and response are still handled manually. Even with SIEM and SOAR platforms in place, a large share of security operations remains only partially automated.

Analysts must handle an ever-growing stream of events from network sensors, cloud platforms, and endpoint agents. This overload leads to delays in incident investigations, and some attacks get lost in the noise of the overall event flow.

Autonomous AI assistants help ease this burden by taking over routine tasks. In practice, they already:

• Perform initial event classification;
• Correlate telemetry from different sources while factoring in infrastructure context;
• Trigger standard response actions, from enriching Indicators of Compromise (IoCs) to isolating affected nodes and updating response rules.

In DevOps, the shift to an agent-based model is happening even faster. Automation is already deeply embedded in the Software Development Life Cycle, so autonomous systems fit naturally into CI/CD pipelines. Where AI once helped developers generate code or detect bugs, agents now assume responsibility for managing the build, test, and deployment stages of the software lifecycle.

 

Why Initial Risk Models Do Not Apply

In the early era of LLMs, the main risks were centered on data: leaks through chat conversations, poisoning of training datasets, or the generation of harmful content. As a result, defenses focused primarily on controlling requests and filtering responses.

With the rise of AI agents, this model has become outdated. An agent is not just an interface for interaction; it is an entity with delegated authority. It acts on behalf of a service account and holds access rights to APIs, cloud resources, code repositories, and CI/CD pipelines.

As a result, attackers have shifted their focus. The main objective is no longer to provoke a malicious output but to take control of the agent itself, a tactic often described as goal hijacking. Industry observations indicate that most attacks on autonomous systems target the extraction or manipulation of internal instructions. Once decision logic is compromised, attackers can leverage the system’s legitimate permissions to escalate their actions further.

In a SOC environment, a compromised agent may not leak data directly. Instead, it can manipulate or remove events from logs, masking the real attack. For example, it might delete all references to a specific IP address from the past 24 hours, effectively making the intrusion invisible.

In a DevOps environment, a compromise does not necessarily expose source code. Rather, it undermines the integrity of build artifacts and software components, affecting the entire supply chain. A vulnerable Model Context Protocol (MCP) server, for instance, could inject a dependency that contains a hidden remote-control mechanism into the build process. Trusting the source, the agent incorporates the compromised component, leading to contamination across the pipeline.

The scale of impact also changes. Incidents that once resulted mainly in localized data exposure can now pose direct operational risks, including service outages, supply chain disruptions, and disrupted response workflows. Security teams must shift their focus from asking what the agent said to examining what it did, whether it was authorized to do so, and the context in which it did so.

 

Common Ways AI Agents Are Attacked and Compromised

The OWASP Top 10 for Agentic Applications 2026 formalizes how security risks manifest in autonomous AI systems, focusing on agents as operational actors with delegated authority and infrastructure access. Rather than examining isolated model outputs, the framework analyzes how agent behavior, tool use, identity, and coordination can be manipulated or degraded. Despite varied scenarios, these risks can be aligned with three strategic attacker objectives:

 

Objective 1: Goal Hijacking

Goal Hijacking (ASI01) is an evolution of classic prompt injection adapted to autonomous environments. Instead of targeting a direct user request, attackers inject malicious instructions into the task context that the agent already trusts.

These attacks rely on indirect prompt injection, in which harmful directives are disguised as legitimate data in sources the agent consults during execution. Examples include documents retrieved through RAG systems, help desk tickets, code comments, emails, or operational reports. Because this information appears to be part of the working context, the agent may alter its objective or sequence of actions without detecting an anomaly.

The EchoLeak incident (CVE-2025-32711) illustrates this pattern. A crafted document delivered through normal collaboration channels triggered unauthorized data transfer via Microsoft Copilot, not because of a model flaw, but because the system treated embedded instructions as valid context.

Detection is difficult because payloads are often embedded in structured formats such as JSON, YAML, or configuration files, which bypass semantic validation.

Mitigation requires tightly defined task objectives, lifecycle enforcement of intent, and semantic guardrails that detect behavioral drift regardless of input source.

 

Objective 2: Abuse of Trusted Privileges

The second objective is to weaponize the agent’s legitimate access against its own environment. OWASP classifies this risk as Tool Misuse and Exploitation (ASI02) and Identity and Privilege Abuse (ASI03). Agents routinely operate through privileged interfaces, including cloud APIs, command-line tools, email platforms, repositories, and CI/CD systems. Attackers do not need to bypass authentication. Instead, they manipulate the agent into using approved capabilities in harmful ways.

In one Amazon Q case, malicious instructions were embedded in a pull request that the agent reviewed as part of its normal workflow. The content appeared to be a routine technical comment, yet the agent interpreted it as an operational directive and attempted to delete cloud resources through the AWS CLI. No credentials were stolen and no infrastructure vulnerability was exploited. The attack succeeded by misusing trust rather than bypassing controls.

Related risks extend to Agentic Supply Chain Vulnerabilities (ASI04), where trusted components such as MCP servers, plugins, models, or datasets are compromised. In one case, a malicious npm package, “postmark-mcp,” impersonated the Postmark service and introduced a backdoor that silently BCC’d emails to an external server for exfiltration. From the agent’s perspective, the tool remained legitimate, so operations continued without interruption.

Mitigation requires tightly scoped, time-bound privileges via Just-in-Time access and the maintenance of an AI Bill of Materials. Without an AI-BOM, organizations cannot quickly identify exposed components or understand the scale of a supply chain compromise.

 

Objective 3: Undermining the Resilience of the Agent System

The third strategic objective is not to seize control directly but to undermine system stability and reliability. OWASP groups these risks under Memory and Context Poisoning (ASI06), Insecure Inter-Agent Communication (ASI07), and Cascading Failures (ASI08).

Memory poisoning differs from a one-time injection because its effects persist. Malicious data is embedded in the agent’s state, interaction history, or knowledge sources, such as RAG stores, and can influence future decisions. Altered entries can cause the agent to ignore indicators of compromise, creating a lasting blind spot.

In multi-agent environments, the impact spreads quickly. If a scheduler agent’s memory is compromised, corrupted instructions propagate to dependent agents. Without authentication and integrity checks on inter-agent messages, cascading failures often occur, often described as multi-agent escalation.

A related risk, Human-Agent Trust Exploitation (ASI09), arises when a compromised agent uses perceived authority to influence users into approving harmful actions, obscuring attribution.

Mitigation requires a dedicated security layer for memory and communications, including integrity monitoring, state auditing, mutual authentication, automated isolation and kill switches, and workflows that enforce human verification for high-risk actions.

 

Defense Strategy: Engineering Control over Autonomous Agents

The question is not whether agents can be trusted, but how authority is delegated to them, scoped, and constrained in practice. Security programs must treat them as managed identities with defined scope, verified dependencies, observable behavior, and revocable access. The objective is to embed control points within the execution path itself so that autonomous decisions do not translate directly into unrestricted action.

The question is no longer whether agents can be trusted but how authority delegation to these systems is constrained in practice.

 

1. Zero Trust for Non-Human Identities

The foundational principle is to adopt a zero trust model. Every agent is treated as potentially compromised, regardless of its role or origin. This removes implicit trust from the environment and requires explicit verification for each interaction with tools, APIs, and infrastructure resources.

In practice, this means enforcing strict separation of access zones. Agents may communicate only with clearly defined services and endpoints. Even within a single SOC or CI/CD pipeline, trust is not shared broadly. Each tool and service is handled as an independent entity with its own access policies and validation requirements.

 

2. Least Privilege and Just Enough Authority

In agent-driven environments, the principle of least privilege is extended by limiting autonomy itself. An agent receives only the level of independence required to complete a defined task and nothing more.

For example, a system may analyze the results of static or dynamic code testing or enrich events with additional context. Yet, it should not make final deployment decisions without human approval. It might initiate node isolation during incident response, but it should not be able to alter access policies or modify network rules.

Sensitive operations rely on temporary, tightly scoped credentials through Just-in-Time access. Every action is validated against its context and intended purpose, which helps prevent misuse of otherwise legitimate privileges.

 

3. Proxy Layer and Guardrails

A mediation layer should sit between the agent and all external tools or APIs, serving as a policy-enforcement point rather than allowing the agent to invoke capabilities directly.
This layer evaluates each requested action in context: what the agent is trying to do, which identity it is using, whether the action matches its assigned task, and whether the sequence of calls is expected.

Guardrails are implemented as executable policies, not guidelines. They validate intent, restrict tool usage to approved paths, and stop actions that are technically permitted but operationally inconsistent with the objective. This prevents agents from misusing legitimate privileges or chaining benign operations into harmful outcomes.

 

4. Ecosystem Control and Component Pre-Validation

All external components that an agent interacts with, including MCP servers, plugins, third-party tools, and models, are integrated only after undergoing prior validation; service virtualization helps test component behavior without exposing live infrastructure.

This creates a controlled supply chain for AI components, much like dependencies are managed in DevSecOps. The origin, version, and integrity of every component are tracked, reducing the risk of supply chain compromise and ensuring that only verified elements become part of the operational environment.

 

5. Behavioral Telemetry and Security Monitoring

Traditional security metrics are insufficient for agent-driven environments. In addition to access logs and network activity, organizations must capture behavioral telemetry, such as tool call sequences, changes in action patterns, and deviations from expected workflows.

This data should be ingested into SIEM and SOAR as a distinct event class, establishing agent behavior as its own signal layer and making AI governance a routine part of monitoring, detection, and response rather than an ad hoc activity.

 

6. Isolation and Sandboxes

Even with preventive controls in place, agents should operate within isolated environments. Running them inside containers or virtual machines with tightly restricted access to networks, file systems, and system resources helps contain the impact of errors or attacks. This isolation serves as a final containment layer, preventing incidents from propagating beyond the agent’s execution boundary.