How AI is Reshaping Enterprise Resiliency

An AI system can be available, responsive, and still be quietly wrong. Data can be altered, a model can drift, and an autonomous agent can act on a flawed assumption hundreds of times before anyone notices. None of that shows up on a dashboard built to answer the singular question: is the system up?

That question defined much of enterprise resilience for thirty years. The non-deterministic attributes of AI are redefining it. Availability is no longer the finish line; the new standard is whether the system is still performing the business function it was trusted to perform.

Simply put, this is not your father’s resiliency plan. Then again, Oldsmobile only changed the slogan, not the car. We cannot just relabel the dashboard and be AI resilient. 

 

What is AI Resiliency?

For this conversation, we’re defining AI resilience as an organization’s ability to prepare for, withstand, and recover from failures involving AI systems, agents, or autonomous workflows. Equally as important, it’s also the ability to adapt continuously as models, threats, and dependencies evolve. The purpose of AI resilience is to preserve critical operations as well as overall trust in AI-driven decisions and actions.

Resilience programs typically focus on four familiar capabilities: anticipate, withstand, recover, and adapt. NIST formalizes them in its cyber-resiliency framework, ISO 22301 applies similar discipline to business continuity, and CSA’s AI Controls Matrix extends it to AI systems.

I learned early in my career that how professionals view resiliency, and their tolerance for risk, can vary greatly.  For one Fortune 2000 organization, the business continuity plan was to keep their critical system backups in the trunk of the system administrator’s car as their “warm site,” not considering the lower security to access or the physical toll of parking outside during Arizona summers.

That may seem ridiculous today but some of the same lack of foresight is being applied to AI deployment.

The reality is boards need assurance that the integrity of the enterprise procedures and continuity will not be compromised unexpectedly with the speed and scale at which AI could possibly deviate.  And those are metrics not commonly addressed in resiliency dashboards today.

 

The Silent-Failure Problem

Traditional incidents usually create a visible signal: a service stops, a certificate expires, or a login fails. By contrast, AI failures can generate silent flawed outputs that are not discoverable or only found at the time of an audit, customer complaint, or regulatory inquiry. 

This is not to say that the problem is entirely new.  While at an acquiring bank, I reviewed hundreds of retail and other merchant compromises where teams responded to so much noise while the incident went undetected. The system had a wonderful resiliency policy that failed to be trustworthy at the moment of crisis.

The most important lesson I took away is to test early and often in highly complex and ever-changing environments.

The first requirement for AI resiliency is observability—not only of infrastructure, but of data, models, outputs, and actions. Organizations must verify provenance, detect tampering and drift, monitor sensitive-data exposure, and trace which model, prompt, retrieval source, and tool call produced an action.

Absence of an alert is not evidence of integrity.

The second challenge is concentration. And mitigation to reduce this risk could come in many flavors.  Most enterprises rent reasoning from a few model providers. An outage, safety failure, or model change can alter many workflows simultaneously. Diversification on paper may still depend on the same model, cloud, or orchestration layer.

Testing then becomes a critical exercise with a regular cadence within the security team.  This includes mapping critical uses and dependencies and being able to determine that all validated decisions along with services can be restored.

 

Three Disciplines for an Operational Lifecycle

1. Recover trustworthy decisions along with service

AI continuity should define how quickly the organization can restore confidence in outputs and actions. That requires versioned models, prompts, and policies; validated fallback models or manual processes; reproducible decision logs; and restoration drills. Recovery objectives should distinguish service availability from trust recovery.

A system brought back online before its behavior is validated is not recovered.

As I was reminded this week at a conference, every consequential agentic workflow also needs a tested means to pause or constrain it. Call it a kill switch, circuit breaker, or safe mode; the label matters less than the capability. Rubrik Zero Labs research found 88% of security leaders want an undo button to roll back an agent’s actions, with low confidence in current capabilities.

Security teams should know who may activate it, what it stops, how the business degrades gracefully, and what evidence is required before autonomy is restored.

Several CISOs have told me that the testing of a kill switch led to discovery that certain AI services were unable to immediately stop and required a series of isolation steps to confidently end a service. 

 

2. Treat the AI supply chain as operational infrastructure

Organizations need an AI bill of materials identifying the models, versions, datasets, fine-tunes, libraries, retrieval sources, orchestration layers, and tools within each critical workflow.

Fortunately, AI is also the best tool for aggregating BOM on a repeatable basis. This allows organizations to have a significantly better perspective of the overall risk. It also allows them to assess an upstream incident’s blast radius or determine which decisions require review.

Sometimes overlooked is the third party's role in notification.  Contracts should require timely notice of model changes and incidents, meaningful service and behavioral telemetry, audit rights where appropriate, and tested exit or portability provisions. A provider’s change to model behavior is a production change to the enterprise. It should trigger evaluation, monitoring, and, for high-impact uses, approval.

 

3. Apply Zero Trust to agents and their actions

Agents are a new type of non-human identity that should receive the same discipline applied to privileged users: unique identities, least-privilege access, short-lived credentials, scoped tools, transaction limits, and continuous verification. Non-human identities already outnumber humans at least 82:1 in most enterprises. High-consequence actions should require stronger authorization, independent validation, or human approval. This is especially true considering a 2026 CSA survey found that 68% of respondents could not clearly distinguish human activity from AI-agent activity.

Segmentation and policy enforcement limit the blast radius when an agent is compromised or simply wrong. The goal is not to eliminate autonomy, but to make autonomy bounded, observable, and reversible.

 

A Board-Level Resilience Agenda

Boards are usually experts in enterprise risk. While they do not require being knowledgeable in the technology, they must govern the ability to detect and recover from failure of the AI investment. Four elements of AI resiliency belong on the standing agenda:

• Autonomy: Where can AI output directly trigger a decision, transaction, or customer interaction without human review?  And what are the potential ramifications?
• Supply Chain Dependency: Which models and providers support those workflows, and can management trace the impact of a vendor or version change?
• Operational Integrity: How would we detect a confidently wrong system, and how long would it take to restore trust (not merely uptime)?
• Business Continuity: If we paused our most important agent tomorrow, would the service degrade gracefully or stop? What would be the business outcome?

Management should report a small set of resilience measures: inventory coverage; time to detect behavioral deviation; time to contain autonomous action; time to restore validated output; percentage of critical agents with tested fallback and circuit-breaker procedures; and concentration across model, cloud, and data dependencies.

 

The First 90 Days

My colleague, Rich Mogull, does a great job at articulating the importance of the first 90 days when building your AI maturity model.

The key is to start with the highest-consequence decisions, not the largest inventory.

Map where AI can act, what it depends on, and who owns the outcome. Establish telemetry for data integrity, behavioral drift, exposure, and agent actions. Give every agent a governed identity and bounded authority. Add model-change and incident obligations to supplier agreements. Then run one realistic exercise that combines a silent model failure, an upstream provider issue, and a decision to suspend autonomy.

I’d be remiss not to mention that the AI Controls Matrix provides an open source and auditable foundation across continuity, model security, data governance, identity, and supply-chain risk.

Frameworks create value but only when converted into tested capability.

AI Resiliency is not the promise that AI never fails.  As Stanford University has reported, the number of AI incidents is unsurprisingly increasing.

Instead, AI Resiliency is the ability to prepare for failure, withstand its consequences, restore trustworthy operations, and adapt before the next event.

Uptime may tell you the system is running. AI resilience will inform you if the enterprise remains in control of the journey.