Industrial Scale AI Distillation is a Cloud Security Failure

 

CISA's AA26-251A should end the fiction that confidential model weights are enough to protect frontier capabilities.

On September 8, 2026, the Cybersecurity and Infrastructure Security Agency, National Security Agency, and Federal Bureau of Investigation released Joint Cybersecurity Advisory AA26-251A. The advisory attributes industrial-scale knowledge-distillation campaigns to six China-based AI companies and says the activity extracted billions of tokens through millions of exchanges involving U.S. frontier models.

For cloud security professionals, the most important detail is the attack path. The reported campaigns did not depend on stealing model weights or breaching a training cluster. They used legitimate-looking channels: API calls, premium subscriptions, remote cloud services, third-party aggregators, proxy networks, and pools of accounts. The industry drew the security boundary in the wrong place.

Figure 1  A student model can learn capabilities from repeated outputs without receiving the teacher model's weights

If a campaign can distribute itself across thousands of accounts and providers, per-account defenses are security theater.

 

THE BASIC CONCEPT: Distillation and unaccountable access

Knowledge distillation is an established machine-learning practice in which a smaller student model learns to reproduce some of the behavior of a more capable teacher model. With authorization, it can reduce inference costs and make advanced capabilities available in smaller systems.

The technique itself is not malicious. The security problem begins when extraction is unauthorized, deceptive, high-volume, or designed to evade safeguards. A single coding request is not an attack. Thousands of coordinated accounts concealing their origin and switching providers when blocked are.

 

THE CLOUD ANGLE: The model boundary includes the delivery system

The effective attack surface includes account creation, identity verification, credential issuance, subscriptions, inference endpoints, API gateways, aggregators, proxy infrastructure, telemetry, and model routing. Responsibilities across that chain remain fragmented. A model provider sees prompts without full origin context. A cloud provider sees network and account relationships without semantic intent. An aggregator sees cross-model routing but may preserve too little attribution.

That fragmentation is an architectural failure. If every participant can plausibly say the decisive signal belongs to someone else, the ecosystem has designed itself to miss distributed abuse.

 

SIGNALS THAT PER ACCOUNT DEFENSES MISS

• Shared accounts appearing across multiple IP addresses or user agents.
• Continuous usage with little human variation or idle time.
• New subscriptions operating at maximum capacity immediately.
• Similar prompts distributed across otherwise unrelated account pools.
• Coordinated pathway switching and sudden disappearance of identifying metadata.

The campaign is a pipeline. Access, obfuscation, routing, extraction, validation, and failover form a pipeline. Bulk subscriptions create scale; transfer stations hide origin; APIs, cloud platforms, and aggregators distribute traffic; chain-of-thought prompts harvest reasoning; quality checks detect countermeasures; and failover keeps collection running when a pathway is blocked.

 

WHAT THE ADVISORY SAYS WAS DISTILLED: The target was the capability stack

The agencies describe campaigns organized around commercially valuable behaviors: reasoning, software work, agentic tool use, model-training methods, user interaction, specialized knowledge, and multimodal systems.

 

CAPABILITY DOMAIN 01: Reasoning and evaluation

The advisory attributes work on chain-of-thought reasoning, mathematical reasoning, quality evaluation, question answering, and response scoring to campaigns involving DeepSeek, Moonshot AI, MiniMax, and Z.AI.

 

CAPABILITY DOMAIN 02: Software and agentic work

Coding, code review, data analysis, tool use, end-to-end agent workflows, and computer-use agents appear across activity that the agencies attribute to DeepSeek, Moonshot AI, Alibaba, MiniMax, and StepFun.

 

CAPABILITY DOMAIN 03: Model building methods

The reporting describes data collection for supervised fine-tuning, reinforcement learning, distillation training, and dataset refinement. Extraction can target the process for building a model, not only its visible answers.

 

CAPABILITY DOMAIN 04: Domain and product behavior

Examples include legal specialization, customer-service dialogue, virtual characters, coaching and assistant behavior, functional creation, and occupational writing. These interaction patterns help make models useful products.

 

CAPABILITY DOMAIN 05: Creative and multimodal systems

The advisory also describes creative writing, computer vision, and computer-use agent development. Distillation can move from text generation into systems that perceive, decide, and act through tools.

 

Why the domain mix matters

The objective extends beyond copying what a model knows. The campaigns seek to reproduce how a frontier system reasons, codes, uses tools, serves customers, and helps train the next model. That is an extraction of product capability and development leverage, not ordinary data scraping.

 

MITRE ATLAS LIFECYCLE AND NOVEL TECHNIQUES: The campaign operates as a managed extraction system

AA26-251A maps the activity from infrastructure acquisition through external harm and separately identifies four techniques that make the campaign cheaper, harder to attribute, and resilient to blocking.

Figure 2  MITRE ATLAS stages and novel operating techniques identified in Joint Cybersecurity Advisory AA26-251A

 

How to read the lifecycle

The MITRE ATLAS entries describe campaign phases. Four novel techniques operate across them. Regional and subscription evasion creates access. Centralized routing and automated metadata sanitization conceal relationships. Quota and cost optimization sustains volume. None is reliably visible in one account, one provider, or one log source.

Defender implication. Correlate identity, billing, network, prompt, routing, and model telemetry across organizational boundaries. Otherwise, each participant sees a legitimate-looking fragment of a coordinated extraction system.

 

THE COST OF GETTING THIS WRONG: The damage extends beyond model intellectual property

The extractor captures a reusable capability. Everyone else inherits the cost of defending, verifying, and trusting the ecosystem around it.

 

IMPACT 01: Economic displacement

The advisory says successful campaigns shorten development timelines and reduce frontier-model training expenses. The extractor gains capability while providers lose market differentiation and absorb fraud, monitoring, investigation, and countermeasure costs.

 

IMPACT 02: Capability without safeguards

A student model may reproduce useful reasoning, coding, or agentic behavior without inheriting the teacher's surrounding safety controls. Powerful capabilities can move into systems with weaker governance, testing, and misuse prevention.

 

IMPACT 03: Cloud ecosystem strain

Platforms and intermediaries absorb higher verification, telemetry, retention, correlation, and abuse-response costs. Legitimate users can face more onboarding friction, false positives, throttling, or degraded service.

 

IMPACT 04: Enterprise supply chain risk

Buyers can inherit models whose lineage is difficult to verify. That uncertainty becomes a procurement, compliance, resilience, and reputational problem, particularly when AI supports regulated or security-sensitive workloads.

 

Who pays for successful extraction

Model providers fund the original research and defenses. Cloud platforms and aggregators absorb monitoring and incident-response work. Legitimate customers encounter added friction, false positives, and the risk of silent degradation. Enterprise buyers inherit provenance and compliance uncertainty. The wider ecosystem absorbs the safety risk when capable models circulate without equivalent controls.

The economics are asymmetric: the extractor keeps the capability while the ecosystem pays to rebuild visibility and trust. Because no single participant sees the complete operation, cross-organization intelligence sharing is a security control.

 

FIVE CONTROLS FOR THE AI ECOSYSTEM: Controls the industry should treat as nonnegotiable

ACTION 01: Stop treating AI access like a growth funnel

High-throughput AI access is privileged infrastructure, not ordinary SaaS onboarding. Providers should not reward minimally verified accounts with immediate maximum usage. Scope credentials to a workload, tenant, environment, and business purpose; use short-lived credentials; rotate secrets; prohibit shared human accounts; and enforce explicit throughput limits. A premium subscription should never become an unexamined enterprise data-generation pipeline.

 

ACTION 02: Make legitimate automation prove itself

Evaluation suites, coding agents, customer-service systems, and batch inference jobs can resemble extraction, so volume alone is a weak signal. Require high-volume customers to register automation patterns, bind them to verified identities and declared use cases, and establish baselines by account type. If a platform cannot distinguish approved batch evaluation from covert extraction, it does not have a usable AI control plane.

 

ACTION 03: Correlate behavior or accept blindness

Blocking one API key is not mitigation when another account can replace it seconds later. Connect users, organizations, credentials, devices, IP addresses, payment instruments, cloud regions, request structure, timing, volume, and provider-switching patterns. Anthropic reported that campaigns it attributed to DeepSeek, Moonshot AI, and MiniMax used approximately 24,000 fraudulent accounts and generated more than 16 million exchanges. That scale makes account-by-account monitoring obsolete. Privacy and data-minimization requirements should shape responsible correlation, not justify signals too fragmented to defend the service.

 

Test the whole campaign

Red-team exercises should distribute similar requests across identities, regions, subscription types, and access pathways. A test that ends after blocking one noisy account validates the wrong defense. Require teams to reconstruct the campaign and prove they can exchange useful indicators with ecosystem partners.

 

ACTION 04: Treat silent degradation as a last resort

CISA's most controversial recommendation is to alter responses to high-confidence malicious extraction activity. Reducing reasoning depth, routing requests to weaker models, output obfuscation, or differential privacy may raise attacker costs. But an opaque defense that harms legitimate customers creates a second trust problem. Silent degradation should be a last resort governed by strict confidence thresholds, independent enforcement review, change approval, rollback, customer-impact testing, and protected pathways for authorized evaluators.

 

ACTION 05: Require provenance before procurement

Model provenance should be an entry requirement, not an optional questionnaire item. Ask which foundational or teacher models contributed to a system; whether third-party outputs were used for training, fine-tuning, or evaluation; what rights governed that use; and whether the provider can document model and dataset lineage. A vendor that cannot answer should not receive the benefit of the doubt merely because its model is fast or inexpensive. AA26-251A should not become an automatic ban list. It should force evidence-based review, especially for regulated, safety-critical, government, or security-sensitive workloads.

 

The industry has been defending the wrong asset

The industry cannot rate-limit its way out of a distributed ecosystem problem. Fraudulent identities, shared credentials, concealed routing, coordinated quota consumption, prompt injection, automated failover, and large-scale output collection are reusable threat patterns.

An AI model exposed through an API is not protected merely because its weights remain confidential. Its capabilities can be queried, measured, collected, and approximated through the cloud services surrounding it. Model providers, cloud platforms, API aggregators, application teams, and customers all need evidence-backed responsibilities.

The verdict. If defenders cannot see coordinated extraction across the system, they are protecting individual accounts rather than the model.

 

Sources and practitioner resources

• CISA NSA and FBI Advisory AA26-251A
• Joint Advisory PDF with domains and TTP detail
• Anthropic Detecting and Preventing Distillation Attacks
• CSA AI Controls Matrix version 1.1
• NIST AI 100-2 Adversarial Machine Learning

Editorial note: Claims about named organizations are attributed to the joint U.S. government advisory.