Industry Leadership
Strategic Initiatives
CSA's strategic programs driving innovation in AI, cloud, and Zero Trust.
A public-interest 501(c)(3) dedicated to secure and trustworthy AI.




Industry Leadership
Strategic Initiatives
CSA's strategic programs driving innovation in AI, cloud, and Zero Trust.
A public-interest 501(c)(3) dedicated to secure and trustworthy AI.

CSAI FoundationChaptersEventsBlog
Cyber Monday: Save 50% on CSA online training and certificate exams for 24 hours on November 30. Get the code →
Publication Tag

Designing the AI-First Security Organization

AI Safety

Designing the AI-First Security Organization
AI is reshaping security organizations in major ways. Businesses are adopting AI that security teams must defend. Security functions are using AI in their own operations. Adversaries are deploying offensive AI.

These forces affect processes, skills, staffing, governance, and team structures. Responding to them requires more than technology adoption.

This publication provides a framework for restructuring security teams around machine-scale capacity. Drawing on CISO discussions from CSA’s AI Storm Summits, the paper maps security functions to seven transformation types. It examines how VulnOps, AppSec, IR, GRC, IAM, TPRM, and other functions consolidate, split, or automate new responsibilities.

The paper recommends security team restructuring before resizing. As agent fleets do more, humans shift from performing routine tasks to validating output, managing fleets, and owning decisions. Learn how to establish AI security governance, define human-in-the-loop boundaries, and avoid common failure modes.

Key Takeaways:
  • The three forces driving security organization change
  • Seven transformation types across eleven security functions
  • How human roles shift toward judgment, validation, and accountability
  • When to apply governance in the loop versus human in the loop
  • Five practical starting points for building an AI-first security organization

Download this Resource

Prefer to access this resource without an account? Download it now.


Best For IconBest For:
  • CISOs & CSOs
  • Vulnerability management leaders
  • GRC, audit, & compliance leaders
  • IAM leaders
  • Security architects & engineers

Executive Summary

AI impacts security organizations in three key ways. As the business adopts AI, security teams need to govern and defend that usage. As security adopts AI for its own operations, those operations change. And as adversaries adopt offensive AI, security teams need to respond to changing risks and threats. Security organizations cannot merely respond with some technology adoption; responding to these disruptions also affects processes, required skills, staffing, and team structures. This paper consolidates the latest CISO discussions and research and provides a framework for building an AI-first security organization.
Frontier AI models are now capable of performing a meaningful share of hands-on security work, especially bounded tasks like vulnerability discovery, code review, threat analysis, and application security assessments. Thus, a security team’s effective capacity has detached from its headcount, and a discrete job function backed by agent fleets can produce the analytic output of a much larger team. Structures built for human throughput are breaking under that output. Queues and approval chains that were merely slow at human scale become the binding constraint at machine scale, yet we also need to refactor processes to maintain human governance at machine speed.

This paper’s central finding, drawn from the AI Storm Summit series and the research base of the CSA AI Safety Initiative, is that the resulting reorganization differs across organizations and across security functions and teams. Each security function experiences a different mix of the three forces above, so each transforms distinctly. Some consolidate, some split, some automate extensively while keeping human approval gates (for high-consequence decisions), and some barely change at all.

What converges is the human role. Across every transformation, humans move from performing the work to guiding and validating agents and owning high-consequence decisions that should not be delegated. Humans manage and direct the fleets that now perform a wide range of autonomous activities. Human time, a scarce resource, is less saturated by repetitive tasks and can focus more on higher-value decisions, judgment, and tasks.

The recommendation is to restructure before resizing, although some organizations may need to surge temporary resources to help respond to adversarial AI risks while transitioning to the new AI-First Security Organization. Redeploy existing staff into validation and fleet-management roles first, then let the evidence drive further sizing. Many organizations will grow, particularly in the net-new functions, while others will hold steady but improve their output and address the long tail of backlog. The window to transition the vanguard functions is measured in months, with one to two years for the broader restructuring. The executive ask is a charter for differential restructuring, with team size decided by the results.

1. The Forces Driving Organizational Changes

Vulnerability exploitation is now the leading initial access vector in attacks, accounting for roughly a third of breaches in the 2026 Verizon Data Breach Investigations Report [2]. Over the same period, frontier models demonstrated that they can operate as capable security practitioners in certain domains. The CSA publication The AI Vulnerability Storm documents the collapse in the cost and elapsed time of vulnerability discovery [1], and this has been further reinforced by ongoing increases in patched vulnerabilities from major vendors like Microsoft (which fixed over 900 issues in their September 2026 patch release). Yet these patches still need to be prioritized, tested, and deployed.

A provocation posed at the summits frames the organizational consequence: if agent fleets make your organization ten times larger by effective capacity, is it organized correctly? Are the humans and the agents structured optimally to avoid issues like the risk of ungoverned agents or human decision overload? While the tenfold figure is hypothetical, the question is still valid. Almost every security organization still carries the org chart and the promotion ladders it built when capacity and human headcount aligned, and this is changing.

Three distinct forces are acting on that structure, and conflating them is why many early reorganizations fail to deliver the desired results (Figure 1).

  • Adversarial AI (D1). Offense at machine speed forces a response tempo whether or not the defender adopts any AI at all.
  • Security’s own AI adoption (D2). Internal use of AI changes how existing functions perform their work.
  • The business’s AI adoption (D3). Enterprise AI use creates net-new work to secure: agents, models, harnesses, and an AI supply chain that did not exist two years ago.

The challenge is no longer how to automate individual security tasks, but how to organize teams whose primary collaborators are autonomous agents. Traditional security operating models assume humans own execution and queues represent capacity. As AI increases effective capacity, organizations need operating patterns that optimize validation, governance, and business decision-making rather than manual throughput.

The pace and structure of the transformations in Section 3 differ because the mix of drivers differs. Vulnerability operations, for example, moves first because adversarial AI (D1) forces the pace and security’s own adoption (D2) supplies proven capability. Governance functions transform deeply on internal adoption alone, without the same external adversarial pressure. Identity is being reshaped mostly by business adoption (D3), which demands, for example, support for agent identities and on-behalf-of authorization patterns.

Figure 1. Three distinct drivers act on the security organization. Their mix varies by function, which is why the transformation is differential.

The change is uneven in a second sense: capability is asymmetric. Discovery is most mature, taking advantage of the excellent search capabilities of AI. Defensive actions such as containment and remediation are improving but remain gated as much by trust and blast radius concerns as by the technology itself. Summit participants consistently described the same position: leaders who can see the old structure breaking and have piloted capable agents, but still have no reference model for what to build instead. They are reorganizing without a map. The timeline they described was consistent as well: months to restructure the vanguard functions where capability is already proven, and one to two years for the broader organizational transition.

Several domains are deliberately limited here and deferred to companion work. OT/ICS receives only a brief catalog entry. Resilience against disruption of the AI vendors enterprises now depend on needs additional research. So does product security, the vendor-side function spanning PSIRT operations, coordinated disclosure, customer security assurance, and update delivery. Its engineering dimension appears inside the application security and vulnerability operations entries, but its customer-facing obligations change enough under machine-speed discovery to warrant their own paper. Physical security convergence and fraud and insider risk programs (beyond the operator-account risks covered in Section 5) are also out of scope as domain- and industry-specific.

2. The Destination, as Options

Three primary operating patterns recurred across the summit series. These are not necessarily mutually exclusive, and can be blended in various combinations:

  • In the product-model security organization, security functions run the way product teams run: prioritized backlogs and internal customers replace undifferentiated ticket queues, and the unit of work is a capability delivered to a consumer of security services rather than a ticket closed.

  • In the engineer-as-agent-manager pattern, the individual contributor’s day focuses on defining and dispatching narrowly scoped agent tasks, then validating results and handling the exceptions the fleet escalates.

  • In the forward-deployed advisory model, senior engineers embed directly in business units, close to the context and the decisions, while agent fleets absorb the queue work that once consumed them. The human interprets what the findings mean for the business in front of them.

These are options rather than a mandate: different functions arrive at different destinations at different speeds, as described in Section 3. What holds across all of them is the human role: whichever shape a function takes, its humans validate fleet output and own the decisions that should not be delegated to a machine. They are supported with functions like Security Champions, security-as-code, and secure-by-default developer tools, which allow higher levels of automation.

The control posture that accompanies these shapes is governance in the loop (GITL): agents operate inside a governed envelope of pre-authorized actions with monitoring and audit, and humans supervise the distribution of outcomes rather than each individual action. Human in the loop (HITL) is reserved for defined high-consequence gates, such as pushing a production update that could cause downtime. Leading-edge adopters treat this as the emerging default, but the summits reached no consensus on exactly where the boundary belongs, and these decisions will vary across organizations based on size, vertical, history, and culture.

One scaling caveat: these patterns describe the options available to a well-staffed security organization. Many organizations run security with a handful of people, and some with a single dedicated security role. Those organizations still need to cover the same functions, at a different scale, with heavier reliance on consolidated roles and external services, supplemented by shared community resources. Appendix A sketches reference shapes at three sizes.

3. How Each Function Transforms

There is no single new shape for the AI-first security organization. What follows is a catalog of function-specific transformations covering the scope of a modern security program (consistent with the NIST Cybersecurity Framework [9]), with CSA’s AI Security Maturity Model (AISMM) overlaid for the net-new work of securing AI itself [10]. Seven transformation types cover eleven functions: consolidation, bifurcation, elevation in place, heavy automation, orchestration over tooling, net-new, and slow or differently shaped. In several places, the catalog deliberately departs from today’s standard org chart, which is the primary objective of this paper. Each entry describes the change and the human role that remains, with a rough pacing assessment.

Function Transformation type Drivers Pace
Vulnerability operations Consolidation D1 + D2 ● Vanguard: months
Application security Bifurcation D1 + D2 ● Both halves accelerating
SOC / detection / IR / resilience Elevation in place D2, D1 pressure ● Fast follower, action gated
GRC / compliance / privacy Heavy automation, accountability-anchored D2 ● Deep change, narrow gate
Third-party risk management Heavy automation, expanded scope D2 + D3 ● Transforming and growing
Awareness / human risk mgmt Heavy automation, expanded scope D1 + D2 + D3 ● Fast follower, edge to center
Identity and access management Bifurcation D3 ● Machine arm urgent now
Network / endpoint / cloud / SaaS posture Orchestration over tooling D2 ● Uncertain, vendor-dependent
Security platform & AI engineering Net-new, inward D2 ● Build now: foundational
Securing the business’s AI Net-new, outward D3 ● Distributed, growing
OT/ICS Slow / differently shaped D1, little D2 ● Resilience-first

D1 = adversarial AI. D2 = security’s own AI adoption. D3 = the business’s AI adoption. Seven transformation types cover eleven functions.

Table 1. Eleven functions, seven transformation types, three drivers. The pace column is the planning input to guide a security leader on where to move first.

Consolidation: vulnerability operations (VulnOps). Vulnerability management, threat and exposure management, patch management, application security assessment, and portions of red team infrastructure testing merge into a single discovery-to-remediation pipeline, referred to here as VulnOps. Similar to DevOps, this describes a continuous, deeply automated, and integrated process. To avoid pull request (PR) flooding that degrades developer trust, VulnOps must interface with software supply chain governance (including SBOMs and dependency trees) to assess the blast radius of library substitutions before issuing automated pull requests. No standard org chart contains this function today, but early adopters are already running versions of it. The defining characteristic is not organizational consolidation alone, but a continuously operating discovery-to-remediation workflow supported by shared validation and prioritization processes.

This consolidation and process change comes directly from continuous AI discovery generating findings faster than separate teams or time-based processes can manage them. Summit participants converged on continuous, multi-stage validation pipelines that add exploit validation to prioritize findings and reduce false-positive fatigue. Humans manage the validation gates, which also include evaluation of compensating controls and sequencing judgment, with HITL for production-disruptive deployments. In Team Topologies terms, this is a complicated-subsystem team: expertise deep enough that it cannot be distributed across the organization, offered to the rest as a service [11]. The need to mature VulnOps and move off traditional patch and vulnerability management is driven by the combination of increased patch release frequency (as providers adopt AI vulnerability discovery), larger and more critical patch sets, supply chain implications, and growing AI-driven adversarial capabilities.

Bifurcation: application security. AppSec’s assessment half joins VulnOps, while the enablement half stays outside and grows. Enablement absorbs findings at the cheapest point, through paved roads and secure-by-default libraries and through developer-facing threat modeling. For example, summit participants found that a large number of AI-discovered findings were of the same class and could be reduced through secure-by-default library substitution rather than individual fixes. Enablement also inherits a mandate that did not exist two years ago: governing AI-assisted development itself, including insecurely generated code and hallucinated dependencies, along with the approved coding harnesses offered to developers. Both halves are accelerating, driven by adversarial pace and internal adoption together.

Elevation in place: SOC, detection, incident response, and resilience. The function keeps its overall shape while the lower tiers hollow out. Agent fleets absorb triage, enrichment, investigation assembly, and detection tuning, while humans concentrate on detection strategy, threat hunting, major-incident command, and potentially disruptive response decisions. Humans also observe and monitor agents for indications of prompt injection and other attacks that target SOC automation. Automated analysis capabilities are maturing faster than secure action capabilities. Containment and other disruptive responses remain behind human approval, and mature deployments treat that gate as a deliberate design decision rather than a default. Scope also extends through recovery, since the security elements of resilience planning and post-incident restoration (the CSF Recover function) belong here. Security resilience becomes more important as automated response raises the cost of error. Expect this transformation to follow closely behind VulnOps as detection and investigation agents mature. Recognize that the SOC is now the place responsible for agent repudiation. Are our agents acting within their boundaries? Did our agent attack another organization in pursuit of its goals? Fulfilling this repudiation mandate requires dedicated agent telemetry: structured execution traces linking prompt context, intermediate reasoning, tool calls, and external outputs. Security platform teams should standardize these logs to give the SOC auditable causality records rather than relying on disparate API gateway logs.

Heavy automation, accountability-anchored: GRC, audit, and compliance. Evidence collection, control mapping, continuous control testing, compliance as code, audit preparation, and privacy operations are among the most agent-amenable work in the security organization. The human gate narrows to attestation and risk acceptance, but the stakes of that gate increase because regulatory exposure and personal liability concentrate on the final signature. Risk registers should address the possibility that internal AI-enabled workflows may perform unauthorized external actions due to compromised inputs, unsafe tool use, excessive permissions, or faulty orchestration. Risk registers must acknowledge that an internal AI may act externally with malice. Internal adoption drives this change without any adversarial pressure required. The transformation is deep even where the reporting line never moves.

Heavy automation, expanded scope: third-party risk management. Periodic questionnaires and point-in-time attestations give way to continuous AI-assisted assessment built on public signals and structured audit artifacts (when available). At the same time, the scope expands, because the vendor portfolio now includes AI providers the business depends on operationally. Models will update in ways that affect output, and may have sustained outages that impact any associated business or security automation. AI may expand this dependency by autonomously using unsanctioned vendors. Contracts need terms updated to reflect AI changes, including new categories of data retention, model change management, rate limits, and digital sovereignty. Concentration risk grows as enterprises converge on a small number of AI (and other SaaS) vendors. At the summits, TPRM stood out as one of the functions with the most open questions on both process and technology. Resilience planning for that dependency, including disconnect and reconnect exercises, is deferred to future companion work.

Heavy automation, expanded scope: security awareness, training, and human risk management. Annual training modules give way to continuous, agent-assisted human risk management, with individualized coaching and just-in-time intervention measured by behavior rather than completion rates (a shift some organizations have already made for phishing training). Adversarial AI makes social engineering cheaper to scale and easier to tailor to the individual, which can be modeled and emulated in training. Internal adoption supplies the coaching and measurement agents. And the business’s AI adoption adds a new mandate: shaping how employees use AI tools safely, including the shadow AI the organization has not sanctioned. Security awareness expands to include AI safety and warnings that AI can and often will attempt to overstep its bounds in pursuit of its goals.

Bifurcation: identity and access management. IAM splits into two arms with different rhythms. Human identity remains the more mature and incremental arm, while machine and agent identity is growing far faster and is already larger: published estimates put non-human identities at forty-five to more than one hundred per human employee, a range that is not directly comparable across sources because they differ in what they count (service accounts, tokens, workload identities, or agents), with AI agents the fastest-growing share [5][6]. Published models still treat IAM as a single pillar, but we see non-human identity requiring different approaches, especially to manage the risks of long-lived and static credentials at scale. Agent identity, attestation-based identity, delegated on-behalf-of authorization, lifecycle, and decommissioning sit on the border between IAM and the security platform and AI engineering functions described below. Agent identity requires governance beyond authentication. Every autonomous agent should have an identifiable owner, defined business purpose, delegated authority boundaries, approval workflow, and auditable lifecycle comparable to privileged human identities. Identity therefore becomes both an authentication problem and a governance problem. To minimize blast radius, agent fleets should operate under zero-standing-privilege (ZSP) models where feasible, relying on short-lived, cryptographically attested workload identities rather than static API keys or persistent service accounts. IAM should retain governance of identity lifecycle and access policy, while Security Platform and AI Engineering provide the shared infrastructure that enables those capabilities. SaaS identity, including OAuth grant sprawl, lands in this arm as well, as do key management, PKI, and secrets.

Orchestration over tooling: network, endpoint, cloud, and SaaS posture. Agents will mostly serve as an orchestration layer over the existing vendor stack instead of a replacement for it. AI will integrate with the tools collecting the data and doing their specialized work, then take over some of the time-consuming human prioritization, analysis, and coordination. How far the function transforms depends substantially on how CNAPP, network, endpoint, and SaaS posture vendors expose agentic interfaces (or APIs), and execution likely stays gated to humans wherever the blast radius of the negative impacts of a change event are high. The scope starts with posture and configuration. Endpoint detection and response operations remain with the SOC, while endpoint hardening, configuration, and agent deployment move into this function. Two structural points: first, the function is already internally specialized by platform, since individual cloud providers, endpoint operating systems, containers and Kubernetes, serverless, and increasingly the SaaS estate each demand distinct expertise. Agent fleets shift the economics of that specialization because platform knowledge can be encoded once in a harness and shared. Second, zero trust binds this entry to identity. As access decisions move from network location to identity and context, the alignment with IAM becomes stronger, and per-request authentication and authorization of agents tightens it further [14]. This remains the least certain entry in the catalog, so design for optionality rather than committing to an option the vendor market may not deliver.

Net-new, inward: security platform and AI engineering. Most larger organizations already staff a security architecture and engineering function that builds and runs the shared tooling. This entry is that function’s next generation, re-centered on supporting AI: harnesses, evaluation pipelines, model gateways, guardrails, cost controls, observability, and the security tooling provided to developers and business teams. Across all three summits, participants described harness architecture as mattering as much as model selection, and warned that automation without evaluation simply produces mistakes faster. Since the summits, model drift and updates have also become growing concerns requiring strategic management. To mitigate indirect prompt injection and unauthorized actions, this function must enforce strict input/output sanitization and deterministic permission boundaries on all tool-calling APIs—including Model Context Protocol (MCP) servers and agent plugins—exposed to security fleets. Platform engineers should structure these harness defenses against the OWASP Top 10 for Agentic Applications, prioritizing mitigations against Agent Goal Hijack (ASI01), Tool Misuse (ASI02), and Identity and Privilege Abuse (ASI03). Additionally, having various sandboxing mechanisms adds a second layer of defense for agent plugins and MCP servers. In Team Topologies terms, this is the platform team: other functions consume agent infrastructure as an internal product instead of each building their own [11]. This function provides the SOC with the audit trails responsible for providing traceability. Additional emergent agent behaviors must be considered in platform engineering. Two examples: first, any shared writable store your agents touch should be treated as attacker-persistable, so assume killing a session or deleting an artifact does not end a campaign, and monitor for cross-agent coordination, not just per-agent misbehavior; second, an agent with no internet plus a service with internet equals an agent with internet.
The fleet’s execution substrate should also be continuously assessed for drift, privilege expansion, unsafe configuration, and control-plane integrity. Agent-generated assessments may provide continuous coverage, but the control plane and periodic independent human review must remain authoritative so the agent cannot certify its own security posture.

Net-new, outward: securing the business’s AI. The AISMM overlay covers the work created by enterprise AI adoption: agentic application security, model security, AI supply chain security spanning MCP servers, skills, and plugins, and AI-specific detection [10]. Securing the AI supply chain requires controls beyond traditional software dependency scanning: model provenance verification, dataset integrity auditing, safe serialization enforcement (such as SafeTensors over pickle formats), and specialized machine learning bill of materials (ML-BOM) tracking. The summits and CSA’s research point in the same direction on structure. This work distributes across the transformed functions, coordinated by a thin spine, rather than concentrating in a standalone team, which would only recreate the silo problems the rest of this paper works to remove. However, at least one team or leader needs decision authority to break through possible organizational disagreements.

Slow or differently shaped: OT/ICS. Individual attackers no longer need domain expertise to threaten industrial environments due to the capabilities of AI. Defenders in those environments cannot patch or upgrade on modern timescales, so resilience and segmentation matter more than discovery velocity, and identity controls carry more of the load. In many industrial companies, the function also sits partly outside the CISO organization, which complicates any restructuring. This entry is brief by design: the domain needs dedicated, prescriptive guidance and is deferred for additional work. Many OT-centric organizations, such as utilities and hospitals, are also resource-constrained. The scaling caveats in Section 2 and Appendix A address them.

Four cross-cutting shifts span the catalog. Threat intelligence becomes a shared agent layer feeding VulnOps and the SOC rather than a separate reporting stream. While threat intelligence is consumed as a shared capability, ownership of intelligence validation, prioritization, and strategic analysis should remain clearly assigned to avoid diffused accountability. Security architecture gains centrality, because someone must design the decision rights and the shared platform the fleets run on, in partnership with security platform and AI engineering. Data security acts as both enabler and control point: fleets need clean, governed data, and security ensures only the right agents gain access to data they are authorized for. SaaS security, often raised as a missing function, distributes into three places: its identity dimension lands in IAM and its posture management in the orchestration entry, while its vendor dimension goes to TPRM.

Collective Defense and Shared Resilience

The preceding sections focus primarily on how AI transforms the internal security organization.

AI also changes the economics of collective defense. As adversaries gain the ability to discover vulnerabilities, correlate exposures, and develop attacks at machine speed, no single organization can observe enough of the threat landscape to defend optimally in isolation. AI increasingly shifts the limiting factor from collecting telemetry to reasoning over it, allowing defenders to synthesize vastly larger volumes of operational data into actionable intelligence.

This elevates operational telemetry, forensic evidence, and validated threat intelligence from supporting artifacts to strategic assets. Security organizations should therefore architect for trusted participation in collective defense through real-time sharing of telemetry or aggregated telemetry, and timely sharing of forensic evidence, indicators, and lessons learned. High-quality shared operational data improves manual and automated threat hunting and understanding of the shape of normal operations and the shape of adversary behavior. This enables improved prioritization, accelerates detection engineering, reduces duplicated effort, and enables more coordinated responses across organizations and across sectors.

New approaches and guidelines will need to be developed to account for data-sharing legal requirements, validation of trust groups, controls to detect and prevent adversary injection into data streams, and data cleansing to meet privacy and other regulatory requirements.

Collective defense consequently becomes a continuously operating capability that synthesizes shared telemetry, forensic evidence, vulnerability intelligence, operational context, and geopolitical awareness into prioritized decisions for VulnOps, threat hunting, detection engineering, the SOC, resilience engineering, executive leadership, and strategic planning.

4. Governance and Decision Rights

The summits kept returning to one governance question: who answers for an automated action that halts a business function? The recommended approach starts from that core question. GITL is the default: agents act within a governed envelope of vetted and tested pre-authorized actions and, where possible, classes. GITL still includes continuous monitoring, sampled validation, and audit trails for the evidence, decision, and action taken, while humans supervise the distribution of outcomes and look for signs of outliers and rare risks sampling may miss. HITL is reserved for potentially destructive decisions, decisions that require accountability, and decisions where human judgment matters most. In practice, the mandatory gates cluster around five categories. Actions that are irreversible, production-disruptive, externally visible, usable for privilege escalation, or subject to regulatory attestation require a named human authorizer (Figure 3).


Figure 2. Governance in the loop is the envelope, and human in the loop is the reserved exception. The placement of the boundary is an explicit, documented, organization-specific decision.

Enforcing this boundary requires an independent policy decision point outside the agent. Because agents can misclassify their own actions under prompt injection, tool-chain composition, or goal drift, the platform must evaluate each proposed tool call against deterministic allowlists and policy rules before execution. Relying on an agent to self-police its own authorization envelope recreates the confused deputy problem at machine speed.

Applied indiscriminately, HITL recreates the human chokepoint the restructuring exists to remove, because every gated action is throttled to human speed. Applied too sparingly, it delegates accountability to a fleet, which no regulator or board will accept. The summits reached no consensus on exact placement, and this paper does not manufacture one. The boundary is organization-specific and use-case-specific, a function of risk tolerance and regulatory posture, and of the particular process the decision is part of. The requirement is to draw it explicitly and document it, and then to staff the gates it creates, instead of discovering the boundary during an incident.

This governance structure maps directly to the NIST AI Risk Management Framework (AI 100-1) and its Generative AI Profile (AI 600-1) [15]. The Govern function defines the policy boundaries, Map catalogs agent identities and tool reach, Measure tracks validation accuracy and error rates, and Manage handles incident response when agents drift or fail.

Table 1 shows example boundaries for representative workflows. Note that the fleet is Responsible everywhere and Accountable nowhere. These are examples only.

Workflow Agent fleet (Responsible) Human owner (Accountable) HITL gate
Vulnerability triage Deduplicate, enrich, tag reachability, validate exploitability, propose severity Accepts or rejects validated findings Confirmation of highest-severity findings before entering the fix queue
Fix deployment Generate fix, produce regression evidence, stage rollout Approves change class, monitors rollout Production deployment outside pre-authorized change classes
Incident containment Collect, correlate, build timeline, draft findings Validates the narrative, sets scope Containment that disrupts a live business function
Detection promotion Propose, backtest, and tune detections Approves promotion to production Any detection that can trigger a disruptive automated response
Compliance evidence Gather, map to controls, assemble packages Attests to accuracy and completeness Human sign-off on anything submitted to an auditor or regulator

Table 2. Decision rights for representative workflows under a GITL-default model.

For automated fix deployments, shifting from HITL to GITL for minor change classes requires more than passing static unit tests. GITL deployment pipelines should incorporate real-time canary telemetry analysis and dynamic regression boundary checks before fully committing automated changes to production.

Governing the fleets themselves is the remaining piece. The foundational control is an agent inventory that answers six questions: what exists, who created it, who provided its authority, what it can reach, what it is allowed to do within that authority, and how to detect and then stop misbehaviors (e.g., kill switches). Least privilege, continuous verification, behavioral monitoring, and formal decommissioning all follow. Building this inventory requires active discovery alongside passive registration. Security teams must identify unmanaged agents, ephemeral sub-agents, and SaaS-native AI features by inspecting identity provider OAuth grants, API gateway token exchanges, and developer harness configurations before policy enforcement can take hold. CSA’s research on non-human identity and agentic AI governance describes the vacuum most organizations are operating in and provides a starting framework [5]. Structurally, this work belongs to the IAM machine-identity arm and to security platform and AI engineering, which is one reason both appear in the catalog as urgent.

5. The Human Transition

Whichever transformation a function undergoes, the human contribution converges on judgment, and moving people there is the hardest part of the program. The SANS 2026 Cybersecurity Workforce Research Report reads as external confirmation of what the summits surfaced: teams are restructuring around capability rather than headcount, and validating AI output has become a core professional skill in its own right [4].

Validation judgment deserves precise definition, because it is the load-bearing skill of the entire operating model. It is the ability to assess whether an agent’s confident, well-formatted output is trustworthy without redoing the work, and it depends on domain depth: a reviewer who cannot reason independently about exploitability has no basis for accepting or rejecting what a fleet reports. Validation increasingly includes evidence provenance: confirming that agent-generated findings remain traceable to original telemetry, reproducible, and suitable for audit rather than relying solely on synthesized summaries. The primary operational risk to this skill is automation bias—where operators face queue pressure and default to rubber-stamping plausible agent outputs. Left unaddressed, rubber-stamp validation degrades human gates into cosmetic friction and halts the development of true domain expertise in junior practitioners. Practical indicators include override rates that fall toward zero, review times that shrink as queues grow, sampled re-validation that finds defects the original reviewer missed, and recurring acceptance of recommendations later shown to be wrong. This also creates a second-order problem: wrong or merely decorative approvals become feedback signals, potentially reinforcing bad conclusions and pushing agent systems in the wrong direction rather than improving them. Mozilla’s AI-assisted discovery of 271 vulnerabilities in Firefox illustrates the need. Past the discovery step, someone still had to confirm each finding, rank them against one another, and do it quickly and efficiently [3].

At fleet scale, validation is also an architecture rather than only a skill exercised on each artifact. Each agent verifies its own output against evidence as a required step before handing it up, the receiving agent re-checks against the evidence itself rather than the sub-agent’s report of it, and a reviewer with fresh context and no exposure to the author’s reasoning gates the final artifact. Two properties decide whether a layer works. First, independence should be specified by whose enumeration of the problem is used, not by how many people look, because two reviewers who share one mental model are one reviewer. Second, verification has to be available in practice and not merely permitted: the evidence and the means to re-derive it must ship together, or review degrades into reading under load. Plausible-but-wrong output is the failure this guards against, since it is well structured, internally consistent, and survives careful reading. Three techniques help, in increasing order of value: re-derive rather than re-read, by re-running the query or recounting the artifacts; force a positive signal, so the check makes the system emit proof instead of merely failing to show a violation; and name the proposition a check actually proves, then ask whether it is the one that matters. A green result from the wrong proposition is more dangerous than a red one, because a red prompts investigation and a green ends it. Any validation step that matters should also be mechanized, since a step that lives in prose is an intention rather than a control.

The same concentration of capability raises the stakes of insider threat and account compromise. A person who manages agent fleets can act at machine scale, so a compromised or malicious operator account is a larger event than one analyst’s. Controls should therefore cover the humans who operate agents as well as the agents: phishing-resistant authentication and just-in-time elevation for operator roles, separation between those who define agent authority and those who approve it, two-person approval for changes to fleet policy and kill-switch settings, and logging of operator actions to the same standard as agent actions.

One nuance keeps the convergence claim precise: the nature of the work converges while career tracks diverge. Some practitioners will build and operate the defensive AI substrate, others will run agent-augmented functions, and the two paths need distinct development and compensation logic even though both center on judgment.

The transition also has a generational cost that summit participants raised repeatedly, and SANS quantifies: entry-level analyst roles, from SOC triage through threat intelligence and incident response (IR) support, are the most affected [4]. Those queues were the profession’s apprenticeship. Organizations that automate the bottom of the ladder without rebuilding a path onto it are eliminating the pipeline that produces their future senior staff. Deliberate replacements exist: rotations through validation seats under senior review, paired investigation of fleet escalations, structured mentorship, and time-boxed capability-uplift engagements. Uplift engagements should be temporary by design, in the mode of a Team Topologies enabling team [11]. An enablement team that becomes a permanent dependency has turned into another bottleneck.

Competency models should be adapted rather than invented from scratch. Anchoring new role and skill definitions to the NICE Workforce Framework and the European Cybersecurity Skills Framework (ECSF) keeps them legible to HR systems and the labor market [12][13]. What changes is the assessment. Graduated certification should measure demonstrated performance across orchestration fluency, validation judgment, escalation calibration, and domain depth, observed on real work instead of inferred from course completion. Assessment should also differ by transformation type: the validation skills a GRC team needs are not those of a VulnOps cell.

Burnout in this transition is an operational risk rather than a theoretical HR statistic. The CISO MindMap 2026 places team burnout among the top concerns for security leaders [7], and the summit pattern behind it was specific: the human who becomes the validation bottleneck absorbs escalating pressure until they leave, taking scarce judgment with them. This is the strongest argument against cutting staff during the transition, whatever the team’s eventual size. An organization that sheds experienced people while asking the remainder to validate machine-scale output risks dismantling the very judgment capacity the new model depends on.

6. Failure Modes

Nearly every failure mode observed across the summit series reduces to the same root cause: human-throughput structure applied to a machine-scale workforce; others are technical risks that the fleets themselves introduce. The catalog below is deliberately short and diagnostic. Each anti-pattern names the transformation it most directly threatens, and every one of them is cheaper to prevent than to unwind.

Anti-pattern What it looks like What it threatens
The Monolith One broad agent, or one team, tasked with everything, so output reliability degrades as scope grows VulnOps, security platform and AI engineering
The Human Chokepoint Every action queues behind one approver, so throughput collapses to human speed and the approver burns out VulnOps, SOC elevation
Headcount-Shaped Thinking Budgeting, measuring, and organizing by seats while bolting agents on top The entire catalog
Ungoverned Sprawl Agents without inventory, identity, or lifecycle, so no one can answer what exists IAM machine arm, security platform and AI engineering
FIFO Triage Machine-scale findings processed in arrival order by a human queue VulnOps, SOC
Token Burn Fleet spend with no outcome metric, mistaking volume for value Security platform and AI engineering, GRC
Credential Inflation Competency ranks awarded for tool familiarity rather than demonstrated judgment The human transition
Data Boundary Failure Agents access sensitive data beyond approved purpose or jurisdiction GRC, privacy, IAM
Automation Bias Operators approve plausible, well-formatted agent output without checking independent evidence or challenging findings, turning human gates into cosmetic sign-offs. The human transition, GRC, VulnOps, SOC elevation
Tool-Hijacking / Indirect Injection Agent fleets executing unvalidated tool calls (MCP/plugins) triggered by untrusted log data, code comments, or external artifacts Security platform and AI engineering, SOC, VulnOps
Invisible Workforce Agents acting without traces (OTEL), or belief that existing systems (EDR, SIEM) will serve as an audit trail. Observability, auditability, repudiation in the case of an agent attacking a third party
Boundary Collapse Agent fleets or their pipelines extend across IT/OT boundary to give agents plant context, defeating segmentation as a control OT/CPS, Security platform, SOC, AI engineering
Read-only illusion Read-only agents access to cyber-physical environments treated safe, but their hit on the fragile devices itself is disruption risk OT/CPS, VulnOps, SOC
Provider Monoculture All functional fleets depend on a single model vendor API with no fallback routing, turning upstream outages into enterprise-wide security outages VulnOps, SOC, GRC, TPRM
Unfunded Substrate The platform exists but nobody is resourced to keep it correct, so validation seats quietly turn into maintenance seats and the fleet degrades without raising an error The entire catalog
Silent Skip The agent omits a required step, such as a mandated lookup, and answers anyway; the audit trail shows a verdict but no call VulnOps, GRC evidence, any function where humans supervise outcomes rather than actions
Control-Plane Coupling The same fleet, identity plane, or model gateway is used to operate controls and to approve, monitor, or investigate its own actions Governance, IAM, SOC
Authorized Misuse via Context Poisoning An agent with valid credentials makes a technically authorized call that is contextually catastrophic after ingesting poisoned context, so access controls see an authenticated principal and let it pass VulnOps, incident containment, IAM

Table 2. The anti-pattern catalog. Diagnostic use: if a symptom in the middle column is visible, the transformation in the right column is at risk.

7. Getting There

Start where capability is proven. For most organizations, the first move is a bounded AppSec and vulnerability-operations pilot: one meaningful segment of the estate under continuous discovery, with a multi-stage validation pipeline that requires confirmed exploitability before anything reaches engineering. Measure absorption rate (defined below) from the first week. The pilot proves the operating model and surfaces governance friction early, while generating the evidence the executive case needs. Organizations that have already proven AI-assisted alert triage or incident response in the SOC can reasonably start there instead, since alert fatigue is a long-standing bottleneck and incidents will occur however well AppSec and VulnOps perform. The principle is to begin where capability is proven, not to prescribe a function.

Measurement should stay simple at the top. The north-star metric is validated risk reduction per security FTE: confirmed, evidence-backed exposure removed per human employee. It rewards efficiency and cannot be gamed by hiring, but it can be gamed through the denominator: moving staff into platform engineering, handing discovery to a managed provider, or reclassifying roles improves the ratio without reducing risk, and shedding staff does the same, which is the move Section 5 argues against. Define security FTE once at the start of the program, report any change to that definition alongside the number, and report the share of platform time spent keeping the fleet correct, since that effort consumes headcount without reducing risk directly. Treat the metric as valid only while validation capacity is not the binding constraint. Precise measurement is difficult and may prove unrealistic. In practice, organizations operationalize it through auditable proxies, chiefly absorption rate (findings resolved divided by findings created), the validated-output rate of the fleets, cost per validated finding, and time from discovery to remediation. These proxies fall into two classes that should not be inferred from one another: ranking quality (validated-output rate, cost per validated finding) and operational outcome (absorption rate, time to remediation). A gain in ranking quality does not by itself show a gain in analyst workload or time to containment, which requires field measurement against a control queue.

Pair them with countermeasures so efficiency is not read in isolation: severity-weighted residual risk, escaped vulnerabilities and recurrence, false-negative rate, validation-queue wait time or escalation backlog age, and a fleet safety measure such as the rate of incorrect actions, rollbacks, policy violations, or human overrides. To prevent Goodhart’s Law—where fleets clear high volumes of low-severity findings to inflate absorption figures—organizations should pair absorption rate with Mean Time to Remediate (MTTR) specifically for High/Critical reachable exploits, ensuring teams prioritize high-impact risk reduction over high-volume trivial fixes.

Absorption rate also falls mechanically when discovery scales faster than remediation capacity, and rises if scanning is reduced, so report it alongside discovery volume and coverage, or against a baseline discovery rate. In cyber-physical environments, many findings cannot be remediated at all (no patch, an unsupported vendor, or revalidation that costs more than the risk), so track those separately as compensating-control or accepted-risk items rather than counting them as unresolved.

The migration itself is phased and gated. Prove the pilot first, then stand up the security platform and AI engineering along with the governance envelope, since both are prerequisites for scale. From there, extend to the fast-follower functions on the catalog’s pacing, and then take on the broader restructuring, including the incentive and competency changes of Section 5, on the one-to-two-year horizon. Each phase needs entry and exit criteria and a rollback position. Reorganizations conducted under duress, after an incident forces the issue, rarely succeed.

The executive case rests on a standard-of-care argument rather than fear. Regulators are already moving: SANS documents regulatory pressure as a dominant driver of security hiring and restructuring, with demand for new specialist roles roughly doubling year over year and personal executive liability rising [4][8]. The credible version of the case acknowledges that the pace of capability change is not predictable, and argues for structural adaptability for exactly that reason. On budget, cost shifts more than it grows: spending tilts from across-the-board headcount growth toward platform engineering, model consumption, evaluation infrastructure, and reskilling. Where teams do grow, growth should follow the catalog, into the net-new functions and the validation seats, not into the queues the fleets are absorbing.

For smaller organizations, the sequence compresses, but the logic holds: cover the same functions at smaller scale, buying rather than building where possible and leaning on shared services and community resources. Appendix A sketches what that looks like.

Five starting points, in order:

  1. Charter the bounded AppSec/VulnOps pilot.
  2. Name an owner for security platform and AI engineering, even if it begins as one person.
  3. Draw and document the HITL boundaries for major functions before the fleets scale, and staff the gates they create.
  4. Baseline initial success metrics, based on desired security outcomes, however imperfectly, so the transition has a way to determine if it is working or not.
  5. Align the first agent-manager role descriptions to NICE or ECSF so hiring and reskilling can start against something legible.

Appendix A: Reference Shapes at Three Sizes

These are reference sketches, not prescriptions. The table condenses the Section 3 catalog: cyber-physical systems (OT/ICS, building and data center systems, and medical devices) have their own row; GRC and TPRM share a row because they converge on shared evidence collection and agent tooling at every size, although Figure 2 treats them as distinct functions, and threat intelligence appears as the shared layer it becomes. Size is only a rough proxy: a large enterprise can run a single governance function with shared agent tooling, and a small one can federate. What changes across these shapes is how centralized the operating model is, how many distinct teams cover the functions, and how much is delivered through external services.

Function Large enterprise Mid-size Small (one to five security staff)
Vulnerability operations Dedicated cell running agent fleets Combined VulnOps and AppSec team Managed discovery service, with an internal owner who validates anything that reaches production and sequences remediation
AppSec enablement Paved-road team partnered with platform engineering Same team as VulnOps, enablement-led Secure defaults adopted from framework and vendor guidance
SOC / detection / IR Elevated SOC with a human strategy layer over fleets Hybrid SOC with managed detection and response (MDR) augmentation MDR or managed security service provider (MSSP) with a named internal escalation owner
GRC and TPRM Separate automation programs Single governance function with shared agent tooling Compliance automation service, while attestation stays with leadership
Awareness / human risk Dedicated human risk program with behavioral measurement Combined with GRC or SecOps Vendor-delivered training plus agent-assisted coaching
IAM Split human and machine-identity arms One team with an explicit machine-identity backlog IdP-centric management, plus a basic agent inventory kept current
Network / endpoint / cloud / SaaS posture Platform-specialized teams under one posture program Combined infrastructure security team Provider-native posture tools, managed where possible
Security platform and AI engineering Dedicated platform team Part-time platform role within security engineering Managed vendor-provided agent capabilities and emerging community patterns, plus a named owner for the minimum substrate they do not supply (inventory, gating, and logging)
Securing the business’s AI Thin coordinating spine, work embedded in functions One named coordinating owner Folded into the general security role, with AISMM as the checklist
Threat intelligence Shared agent layer feeding VulnOps and SOC Curated feeds tuned by the SOC Community and ISAC feeds
Cyber-physical systems (OT/ICS, building, data center, medical) Dedicated cyber-physical security team partnered with plant, facilities, and engineering Named cyber-physical owner coordinating with operations and facilities Asset inventory and segmentation of physical systems, with the vendor or integrator accountable for monitoring

Table 3. Reference shapes at three sizes.

The right-hand column still describes a restructuring. The single security professional in a small organization becomes an agent-manager and a validator of external services’ output, following the same convergence described throughout this paper at minimum scale.

References

[1] Cloud Security Alliance, The AI Vulnerability Storm: Building a “Mythos-ready” Security Program, 2026. https://labs.cloudsecurityalliance.org/wp-content/uploads/2026/05/mythosreadyv1.0.pdf

[2] Verizon, 2026 Data Breach Investigations Report. https://www.verizon.com/business/resources/reports/dbir/

[3] Mozilla, “The zero-days are numbered.” https://blog.mozilla.org/en/firefox/privacy-security/ai-security-zero-day-vulnerabilities/

[4] SANS Institute, 2026 Cybersecurity Workforce Research Report. https://www.sans.org/white-papers/2026-cybersecurity-workforce-research-report

[5] Cloud Security Alliance, Non-Human Identity and Agentic AI Governance, 2026. https://labs.cloudsecurityalliance.org/research/csa-whitepaper-nonhuman-identity-agentic-ai-governance-v1-cs/

[6] Palo Alto Networks, 2026 Identity Security Landscape Chapter One. https://www.paloaltonetworks.com/idira/identity-security-landscape-report

[7] Rafeeq Rehman, CISO MindMap 2026. https://rafeeqrehman.com/2026/04/11/ciso-mindmap-2026-what-do-infosec-professionals-really-do/

[8] Industrial Cyber, coverage of the SANS 2026 workforce report’s regulatory findings. https://industrialcyber.co/reports/sans-2026-report-flags-cybersecurity-skills-crisis-putting-critical-infrastructure-and-ot-sectors-at-measurable-breach-risk/

[9] NIST, Cybersecurity Framework 2.0. https://www.nist.gov/cyberframework

[10] Cloud Security Alliance, AI Safety Initiative and the AI Security Maturity Model (AISMM). https://cloudsecurityalliance.org/ai-safety-initiative

[11] Matthew Skelton and Manuel Pais, Team Topologies, IT Revolution Press. https://teamtopologies.com/

[12] NIST, NICE Workforce Framework for Cybersecurity (SP 800-181). https://www.nist.gov/itl/applied-cybersecurity/nice

[13] ENISA, European Cybersecurity Skills Framework (ECSF). https://www.enisa.europa.eu/topics/skills-and-competences/skills-development/european-cybersecurity-skills-framework-ecsf

[14] Cloud Security Alliance, Zero Trust research and the Zero Trust Advancement Center. https://cloudsecurityalliance.org/research/working-groups/zero-trust/

[15] NIST, Artificial Intelligence Risk Management Framework (AI RMF 1.0) (AI 100-1), 2023; Generative AI Profile (AI 600-1), 2024; and ongoing Agentic AI Profile initiatives, 2026. https://www.nist.gov/artificial-intelligence

Premier AI Safety Ambassadors

Premier AI Safety Ambassadors play a leading role in promoting AI safety within their organization, advocating for responsible AI practices and promoting pragmatic solutions to manage AI risks. Learn more about how your organization could participate and take a seat at the forefront of AI safety best practices.

Explore More of CSA

Research & Best Practices

Stay informed about the latest best practices, reports, and solutions in cloud security with CSA research.

Upcoming Events & Conferences

Stay connected with the cloud security community by attending local events, workshops, and global CSA conferences. Engage with industry leaders, gain new insights, and build valuable professional relationships—both virtually and in person.

Training & Certificates

Join the countless professionals who have selected CSA for their training and certification needs.

Industry News

Stay informed with the latest in cloud security news - visit our blog to keep your competitive edge sharp.