A&A: Audit & Assurance

A&A-01: Audit and Assurance Policy and Procedures

Control Specification

Establish, document, approve, communicate, apply, evaluate and maintain audit and assurance policies and procedures and standards. Review and update the policies and procedures at least annually, or upon significant changes.

Auditing Guidelines for Model Providers (MP)

1. Confirm policies exist specifically for model assurance and are version-controlled.

2. Check for cloud-specific policies (e.g., cloud-native security controls, audit logging in cloud ML pipelines).

3. Approval and Ownership: Identify formal sign-off from leadership (e.g., CTO, Chief Risk Officer). Validate assignment of responsibility for maintaining these policies.

4. Communication and Training: Verify whether relevant staff (e.g., ML engineers, data scientists) are trained on audit/assurance procedures. Check for training logs or awareness campaigns on cloud auditing tools (e.g., GCP’s Cloud Audit Logs, AWS CloudTrail).

5. Application and Scope: Ensure policies cover audit of automated model updates, retraining events, and third-party ML libraries.

6. Review Frequency: Check last review date and whether regulatory changes triggered updates.

A&A-02: Independent Assessments

Control Specification

Conduct independent audit and assurance assessments according to relevant standards at least annually.

Auditing Guidelines for Model Providers (MP)

Objective: Verify that the provider subjects model lifecycle and infrastructure components to independent evaluations in accordance with standards.

1. Review External Audit Records: Obtain attestation reports (e.g., SOC 2 Type II, ISO 27001, ISO 42001), if available.

2. Verify Scope Coverage: Confirm model training environments, data pipelines, and deployment infrastructure are within audit scope.

3. Check Assessment Frequency: Confirm independent audits are conducted at least annually and include any material changes (e.g., new model architectures).

4. Trace Remediation from Past Audits: Examine whether previous findings were resolved and verified by independent auditors.

A&A-03: Risk Based Planning Assessment

Control Specification

Perform independent audit and assurance assessments according to risk-based plans and policies, and in response to significant changes or emerging risks.

Auditing Guidelines for Model Providers (MP)

1. Review the documented process for risk-based planning that informs independent audit and assurance activities.

2. Assess whether risk-based planning appropriately accounts for the specific context of the model, including intended use cases, potential impacts, terms of use, and applicable legal and regulatory requirements.

3. Review documented risk registers, assessments, or other artifacts that identify and evaluate risks across the model’s behavior and lifecycle.

4. Examine policies that define the frequency of audits and specify triggers for audits based on significant changes to the model, such as updates, retraining, or shifts in use context.

5. Interview or confirm the roles of teams or individuals responsible for overseeing model governance and ensuring that audit planning remains risk-based and effective.

6. Verify that records of past audits exist and include documentation of actions taken in response to audit findings.

A&A-04: Requirements Compliance

Control Specification

Verify compliance with all relevant standards, regulations, legal/contractual, and statutory requirements applicable to the audit.

Auditing Guidelines for Model Providers (MP)

1. Identify Applicable Requirements (e.g., EU AI Act, GDPR (if personal data is involved), export controls on sensitive models, IP rights for training data): Check alignment with emerging AI regulations (e.g., EU AI Act, NIST AI RMF) and ethical frameworks.

2. Assess Documentation: Review licenses for training datasets and contracts with third-party data providers.

3. Evaluate Processes: Verify formal processes for regulatory tracking to identify how they keep up with changing laws. Focus on regulatory compliance related to model training data, algorithm transparency, bias/fairness, and AI explainability.

4. Verify contracts governing use of datasets and licensing of model outputs.

5. Examine Controls: Assess if technical safeguards and policy enforcement mechanisms are in place (e.g., bias mitigation, dataset consent tracking).

6. Review Third-Party Attestations: If applicable, examine SOC2, ISO 42001, or similar reports.

A&A-05: Audit Management Process

Control Specification

Define and implement an Audit Management process aligned with relevant auditing standards, to support audit planning, risk analysis, security control assessment, conclusion, remediation schedules, report generation, and review of past reports and supporting evidence.

Auditing Guidelines for Model Providers (MP)

1. Review the Audit Framework: Look for procedures governing model-related audits aligned with relevant auditing standards.

2. Assess Risk-Based Audit Planning: Is the model risk profile used to drive audit frequency and depth? Ensure audit management covers model lifecycle stages: training, validation, deployment, and updates.

3. Evaluate Security Control Reviews: Are there controls for data handling, model access, update integrity? Verify assessments for AI-specific risks: bias, robustness, data provenance, and adversarial threats.

4. Determine if technical audits (e.g., red teaming, model explainability tests) are formalized and documented.

5. Examine Remediation Workflows: Is there a mechanism to track and implement remediations?

6. Check Audit Traceability: Ensure all past audit results and model changes are traceable with evidence.

A&A-06: Remediation

Control Specification

Establish, document, approve, communicate, apply, evaluate and maintain a risk-based corrective action plan to remediate audit findings, regularly review and report remediation status to relevant stakeholders.

Auditing Guidelines for Model Providers (MP)

1. Verify that corrective action plans (CAPs) are clearly tied to audit findings, including those related to model explainability, data leakage, or governance gaps, and that remediations are applied at the appropriate level: model (e.g., retraining), data (e.g., curation), or infrastructure (e.g., access control).

2. Assess whether remediation plans prioritize high-risk issues, such as discriminatory outcomes, and appropriately weigh model-specific risks like hallucinations, bias, and drift.

3. Confirm that remediation plans are reviewed and formally approved by accountable roles, such as ML leads and risk officers.

4. Review how audit findings and remediation progress are communicated across relevant teams, including engineering, risk, and compliance.

5. Verify that remediated models are re-tested to ensure issues have been resolved, including testing under adversarial conditions where applicable.

AIS: Application & Interface Security

AIS-01: Application and Interface Security Policy and Procedures

Control Specification

Establish, document, approve, communicate, apply, evaluate and maintain policies and procedures for application security. Review and update the policies and procedures at least annually, or upon significant changes.

Auditing Guidelines for Model Providers (MP)

1. Verify there are documented and approved security policies for APIs and model deployment endpoints.

2. Inspect whether interfaces use strong authentication (e.g., OAuth2, mTLS). Are role-based access controls (RBAC) defined and enforced?

3. Confirm policies are reviewed after events like model versioning, retraining, or platform migration.

4. Check if threat models include interface misuse (e.g., prompt injection, data leakage).

5. Ensure logs are collected for all API activity and reviewed regularly.

AIS-02: Application Security Baseline Requirements

Control Specification

Establish, document and maintain baseline requirements for securing applications.

Auditing Guidelines for Model Providers (MP)

1. Verify that a formal, approved security baseline exists for each model type, covering all phases of the AI lifecycle: data ingestion, training, testing, deployment, updates, and decommissioning.

2. Determine whether models are classified based on risk impact, purpose, data sensitivity, and regulatory exposure, and confirm that baseline security controls are enforced accordingly.

3. Assess whether the security baseline mitigates key AI risks (e.g., adversarial inputs, data poisoning, unauthorized inference) and aligns with applicable regulatory, legal, and industry standards.

4. Examine configuration management artifacts, training/inference pipeline controls, and access policies to confirm that the model security baseline is applied in deployed systems.

5. Verify that model security baselines are reviewed periodically and updated based on threat intelligence, regulatory change, or after significant model updates. Confirm that controls are revalidated post-update.

6. Verify that mechanisms (manual or automated) exist to detect deviations from the security baseline, such as unauthorized configuration changes, access anomalies, or deviations in expected model behavior.

AIS-03: Application Security Metrics

Control Specification

Define and implement technical and operational metrics in alignment with business objectives, security requirements, and compliance obligations.

Auditing Guidelines for Model Providers (MP)

1. Verify that model-centric metrics are defined (e.g., input sanitization failures, drift events, adversarial flag rate).

2. Assess whether security metrics are linked to the ML lifecycle phases (e.g., training, deployment, inference).

3. Confirm metrics are used to guide model promotion or rollback decisions.

4. Verify alignment with AI assurance frameworks (e.g., NIST AI RMF, OWASP ML Top 10).

5. Check whether metrics are used in incident correlation (e.g., model misbehavior during API abuse).

AIS-04: Secure Application Development Lifecycle

Control Specification

Define and implement a secure SDLC process for application requirements analysis, planning, design, development, testing, deployment, and operation in accordance with security requirements.

Auditing Guidelines for Model Providers (MP)

1. Inquiry with Control Owners: Interview AI model development leaders to understand how security requirements are incorporated throughout model training, validation, and distribution processes. Discuss with ML engineers how they implement security controls for data preprocessing, feature engineering, and model architecture design. Inquire about protocols for model versioning, documentation of model cards, and ethical review processes prior to model release.

2. Review of SSDLC/MDLC policies, procedures, and standards (collectively referred to as the policies below).

   • a. Obtain and review the policies to ensure they adequately address risks associated with secure development of AI/ML models.

   • b. Verify that the policies clearly define responsible personnel/roles, designate an accountable official, specify review frequency (annually), and outline events triggering policy updates.

   • c. Confirm the policies are reviewed and updated at least annually or upon significant changes, properly approved, and communicated to relevant stakeholders.

   • d. Verify the policies include essential requirements covering: MDLC process explicitly including all key phases: design, development, deployment, and operations; clear definition of security roles and responsibilities throughout all phases; consideration of applicable regulatory requirements (e.g., data privacy, AI-specific regulations); documented AI-specific threat modeling or alternative risk mitigation strategies; secure coding standards and guidance or compensating controls; open-source component management, including vulnerability scanning and license compliance or compensating controls; integrated vulnerability management for application code, infrastructure, and third-party components (e.g., SCA, DAST) and requirements for securing the notebooks (e.g., Jupyter, Colab); regular security testing including AI-specific testing (e.g., adversarial testing, LLM benchmarking); secure deployment and configuration management practices; and alignment with industry best practice guidelines (e.g., OWASP).

3. Population Verification: Obtain a complete inventory of AI ML models developed or modified during the audit period from model registries and experiment tracking systems. Verify completeness by cross-referencing with training job logs, model architecture repositories, and deployment pipelines. Confirm the population includes all model types (foundation models, fine-tuned models, and multimodal systems).

4. Inspecting Records and Documents: Select a representative sample of AI models and verify documentation exists showing the policies’ security procedures were followed during training and validation. Confirm each selected model underwent adversarial testing, data privacy evaluations, and bias assessments before release. Verify model documentation includes details on limitations, potential risks, and intended use cases. Review implementation artifacts for each selected sample, including: documented threat models; code samples demonstrating secure coding techniques; evidence of open-source component governance (e.g., SBOMs, SCA reports); vulnerability scanning reports, remediation tickets, and patch evidence; logs or reports from security testing tools (e.g., SAST, DAST, IAST, penetration tests); and secure deployment evidence (e.g., IaC templates, CI/CD security gates, monitoring alerts).

5. Verify that the SDLC defines controls across the model lifecycle, including secure training, validation, deployment, versioning, and decommissioning.

6. Assess whether adversarial testing, bias detection, and model robustness evaluations are built into the testing phase. These are critical for AI-specific threats and cannot be captured by standard security testing (e.g., SAST/DAST).

7. Verify that model artifacts (e.g., weights, checkpoints, metadata) are signed, version-controlled, and validated prior to deployment.

AIS-05: Application Security Testing

Control Specification

Implement a testing strategy, including criteria for acceptance of new information systems, upgrades and new versions, which provides application security assurance and maintains compliance while meeting organizational delivery goals. Automate when applicable and possible.

Auditing Guidelines for Model Providers (MP)

1. Verify that an AI/ML-focused security testing process is in place, which needs to cover training pipelines, inference APIs, and deployment environments.

2. Ensure that automated, continuous tests of the model, updated mechanisms, retraining workflows, and provided APIs for customers to use exist.

3. Verify that documentation exists that describes what has changed compared to the previous version.

4. Ensure that adversarial testing (e.g., evasion, extraction, poisoning, misuse) takes place and that findings are properly mitigated.

5. Ensure that security testing aligns with AI/ML-specific best practices and regularly evolves to cover newly emerging threats.

AIS-06: Secure Application Deployment

Control Specification

Establish and implement strategies and capabilities for secure, standardized, and compliant application deployment. Automate where possible.

Auditing Guidelines for Model Providers (MP)

1. Verify that a written policy exists that links model-lifecycle risk (e.g., data provenance, bias, drift) to the deployment pipeline, and check that approvals for model promotion are role-based and documented.

2. Verify a secure CI/CD for model artifacts. For example, reproducible ML pipelines (e.g., MLflow or Kubeflow) with code-to-model lineage, container signing, and automated security gates (SAST/MLSec scans) before promotion.

3. Evaluate Infrastructure-as-Code (IaC) Usage: Review Terraform or Kubernetes manifests to confirm secure configurations (e.g., network policies, role bindings) are automated.

4. Assess Secrets and Credential Handling: Verify integration with key management systems (e.g., AWS KMS, HashiCorp Vault), and review access logs or secret rotation policies.

5. Validate Compliance Automation: Look for automated license checks or export control validations in pre-deploy hooks.

AIS-07: Application Vulnerability Remediation

Control Specification

Define and implement a process to remediate application security vulnerabilities, automating remediation when possible.

Auditing Guidelines for Model Providers (MP)

1. Review the Vulnerability Management Policy: Request documentation outlining scanning frequency, responsible parties, and remediation SLAs.

2. Inspect Static and Dynamic Scanning Processes: ML pipelines use Python/R/Java libraries with known CVEs. Validate use of automated SAST/DAST tools and SBOM generation for AI components.

3. Check for Open Source Component Monitoring: Models often rely on open-source (e.g., HuggingFace, TensorFlow) and these should be monitored for vulnerabilities. Review integration with tools like Dependabot, OSV-Scanner, or Snyk.

4. Assess Remediation Workflows for Inference Services: Vulnerabilities in model-serving APIs could lead to prompt leakage or adversarial access. Confirm that remediation is tested in staging and rolled out to production through CI/CD with rollback support.

5. Sample Remediation Records: Validate timeliness and traceability of the remediation process. Review tickets, patch logs, and change approvals from recent CVE incidents.

AIS-08: API Security

Control Specification

Define and implement processes, procedures, and technical measures to secure APIs. Review and update for any improvements at least annually, or upon significant changes.

Auditing Guidelines for Model Providers (MP)

1. Verify that measures and processes exist to secure APIs, and ensure that they address: adequacy and relevance, authorization for accessing models, secure key management for model APIs, key mitigations against injection attacks or model theft, and protection of training data and output integrity through API controls.

2. Verify that processes and measures are reviewed at least annually and after significant system change.

AIS-09: Input Validation

Control Specification

Validate, filter, modify or block, as necessary, input against adversarial patterns, failure patterns and unwanted behaviour according to organisational policies and applicable laws and regulations.

Auditing Guidelines for Model Providers (MP)

1. Ensure that the MP has AI-specific input validation processes clearly defined and implemented which address adversarial linguistic, token-based, coding, multimodal, and multilingual prompt manipulation risks. Validate that the documentation is accurate and covers the identified threats and chosen mitigations.

2. Verify that technical mechanisms for AI model-level input validation are in place. Confirm that adequate coverage exists for prompt attacks, including token-level manipulations, programming injection, multimodal and multilingual adversarial techniques.

3. Ensure that the MP regularly conducts AI Red Teaming exercises to assess and validate the model robustness against adversarial prompt inputs which cover rapidly evolving attack techniques including multimodal, multilingual, and adversarial token-based manipulations.

4. Examine AI Red Team findings are comprehensively documented and translated into model-level input validation control improvements for properly addressing AI adversarial threats.

5. Ensure the ongoing effectiveness monitoring of AI model input validation controls through specific metrics tailored to adversarial input detection and mitigation.

6. Confirm that periodic reassessments and proactive updates are conducted.

AIS-10: Output Validation

Control Specification

Validate, filter, modify or block, as necessary, output against adversarial patterns, failure patterns and unwanted behaviour according to organisational policies and applicable laws and regulations.

Auditing Guidelines for Model Providers (MP)

1. Ensure that the MP has AI model-level output validation processes and controls defined and implemented, and that they address OWASP insecure output handling, excessive agency, and adversarial manipulation risks.

2. Validate that comprehensive documentation exists and confirm that it covers evolving threats.

3. Verify that technical controls exist for ensuring AI model outputs are in place and validated for unsafe behaviors, addressing insecure outputs (OWASP-related attacks) and excessive or malicious agency.

4. Ensure that MP undergoes regular AI Red Teaming exercises that evaluate the security of model outputs. A particular focus has to be on the detection and mitigation of the OWASP insecure output handling, as well as excessive agency behaviors and adversarial outputs.

5. Review Red Teaming findings and ensure that findings result in continuous improvement efforts for model-level output validation controls, addressing unsafe, insecure, or adversarial outputs.

6. Verify the effectiveness of the monitoring and output validation controls and ensure that metrics are used that address insecure outputs, excessive agency, and adversarial output risks.

7. Confirm that reassessments are regularly conducted and that model-level output validation controls are updated by leveraging emerging AI threat intelligence and AI Red Team findings.

AIS-11: Agents Security Boundaries

Control Specification

Establish security boundaries for agents.

Auditing Guidelines for Model Providers (MP)

1. Determine if the Model Enables Agentic Behavior: Review model documentation to determine whether the model supports agent‑like behaviors (e.g., action planning, API calling, recursive reasoning) that could be leveraged in agentic architectures.

2. Inspect Model-Level Guardrails: Evaluate the model’s internal safety mechanisms (e.g., refusal behavior, output filtering, or equivalent techniques) that serve as the first line of security boundary enforcement.

3. Assess Documentation of Intended Use and Limits: Review model cards and security documentation to verify that stated limitations, refusal scenarios, and restrictions on agentic use are clearly defined and communicated to downstream integrators.

4. Confirm Model Usage Monitoring: Validate that mechanisms exist to log and detect agent‑like usage patterns, such as unusual prompt categories, override attempts, or abnormal usage spikes indicative of misuse.

5. Evaluate Feedback and Continuous Improvement: Assess whether monitoring results and downstream feedback are reviewed and used to refine guardrails, documentation, and recommended practices over time.

AIS-12: Source Code Managemement

Control Specification

Implement source code management practices, such as version control, code review & static code analysis, aligning with the SDLC process.

Auditing Guidelines for Model Providers (MP)

1. Version Control Validation: Verify that version control (e.g., Git) is implemented for all model code, data preprocessing scripts, training pipelines, and evaluation logic. Confirm repository hygiene practices, such as maintaining clear commit history, appropriate tagging, and enforcing protected branches to prevent unauthorized changes.

2. Code Review Enforcement: Verify that mandatory peer code reviews are in place for all training, preprocessing, and evaluation scripts. Review pull request records or approval workflows to confirm that reviews were completed, documented, and approvals are traceable.

3. Static Code Analysis Implementation: Examine the use of static code analysis tools (e.g., SonarQube, Snyk) to scan for insecure dependencies, configuration errors, and coding flaws in model‑related code. Validate that analysis reports are reviewed, issues are documented, and remediation is completed before deployment.

4. SDLC Integration: Confirm that version control, code reviews, and static analysis practices are formally documented as part of the SDLC (AIS‑04). Identify SDLC checkpoints where model code is gated based on analysis, review, and approval prior to promotion to production.

5. Evidence Review: Review a representative sample of code commit audit trails, code review history (e.g., pull requests, approval logs,) static analysis reports and remediation evidence, and CI/CD pipeline configurations enforcing SCM and security checks.

AIS-13: AI Sandboxing

Control Specification

Implement sandboxing techniques to execute AI tools and plugins in isolated environments to prevent unintended interactions with critical systems or data and limit the possibility of lateral movement.

Auditing Guidelines for Model Providers (MP)

Focus: The AI Model Provider has implemented effective sandboxing techniques to execute AI models, tools, plugins, and extensions in isolated environments, preventing unintended interactions with critical systems or data and limiting the possibility of lateral movement.

1. Inquiry with Control Owners

   1. Interview security architects, research scientists, and ML engineers responsible for implementing sandboxing in AI model execution environments. Obtain and review the organization’s sandboxing policies and implementation standards covering model inference isolation, capability boundary enforcement, function/tool calling security, plugin execution containment, and resource governance for model inference. Verify documented security requirements exist for model inference isolation, function/tool calling boundaries, plugin execution containment, content filtering systems, and code interpreter safety mechanisms.

   2. Review Sandboxing Technical Implementation: Examine documentation describing the technical implementation of container isolation for model serving, function/tool calling validation, plugin runtime environment isolation, network access controls, memory isolation between inference requests, and adversarial prompt protection mechanisms. Assess how input/output validation and resource limitation mechanisms are implemented across different model execution environments.

   3. Assess Plugin System and Tool Calling Security: Review mechanisms implementing security for extended capabilities, including authentication frameworks, execution environment isolation, parameter validation, and capability limitation enforcement for plugins and tools. Evaluate how data access is controlled for plugins and tools, and examine the security review process for first-party tools and third-party plugins.

   4. Evaluate Model Training and Fine-tuning Security: Review procedures for training environment security, including training data isolation, fine-tuning job separation between customers, training infrastructure controls, and model weight protection. Assess security measures for parameter servers, training logs, and the segregation between pre-training and fine-tuning environments.

2. Obtaining and Verifying the Population of Records

   1. Define the Complete Population of Model Components: Obtain a comprehensive inventory of foundation models, serving infrastructure, function/tool calling implementations, plugin frameworks, code interpreters, fine-tuning environments, safety systems, and API endpoints. Include model-to-model integration points, training infrastructure, and model evaluation environments in this inventory to ensure complete coverage.

   2. Verify Population Completeness: Cross-reference the inventory against model capability documentation, API specifications, plugin directories, infrastructure deployment manifests, service architecture diagrams, and developer documentation. Ensure the inventory aligns with public capability announcements, training environment records, and data flow architecture diagrams to confirm completeness.

   3. Categorize Components by Risk Level: Segment the component population based on capability level, data access, external system interaction, plugin support, customization options, fine-tuning access, deployment environment, and usage volume. This risk-based categorization should guide the depth and frequency of security assessment for each component.

3. Inspection of Evidence

   1. Sandbox Implementation Review: Select a representative sample of model components based on risk levels and verify the implementation of isolation mechanisms, resource limitations, and access controls. For isolation, examine container configurations, execution environment separation, memory isolation, and plugin runtime containment. For resource limitations, review token limits, compute caps, memory restrictions, and execution timeouts. For access controls, evaluate authentication mechanisms, permission enforcement, data access controls, and capability access tiering.

   2. Model Capability Security Testing: Review evidence of security testing including prompt injection resistance testing, function/tool calling assessments, plugin isolation testing, code interpreter containment verification, jailbreak evaluations, and data leakage testing during inference. Evaluate the quality and coverage of red-teaming activities and resource exhaustion testing across model capabilities.

   3. Runtime Monitoring and Security Controls: Verify implementation of model behavior monitoring, anomalous request detection, prompt classification and filtering, runtime safety guardrails, and output scanning systems. Assess the effectiveness of function call pattern monitoring, plugin behavior analysis, and resource usage anomaly detection in identifying potential security issues during model execution.

   4. Data Protection within Model Execution: Assess controls for data protection including request isolation between users, prevention of data leakage between requests, ephemeral storage for context, training data access restrictions, and customer data segregation in fine-tuning. Evaluate embedding data isolation, data minimization in logging, and PII handling controls across model execution environments.

   5. Plugin and Tool Security: Examine the implementation of plugin review processes, tool capability restrictions, runtime isolation, parameter validation, and output sanitization for all extensions to model functionality. Assess plugin update verification, integration point security, and security requirements imposed on plugin developers.

   6. Security Incident Response: Review documentation and evidence of model capability bypass incident procedures, prompt injection response playbooks, and customer notification processes. Evaluate how affected model versions are identified, compromised components are isolated, and incidents are investigated. Assess recovery procedures and post-incident model alignment enhancement processes.

   7. Model Alignment and Safety Systems: Verify the adequacy of content filtering implementation, safety system architecture, harmful output prevention, and alignment techniques. Evaluate safety evaluation methodologies, content policy enforcement, alignment testing processes, and system prompt security controls across all model types.

4. Evaluation and Reporting

   1. Sandbox Effectiveness Assessment: Evaluate how well sandbox implementations prevent unauthorized capability access, isolate model execution, contain plugins and tools, limit infrastructure access, maintain appropriate resource allocation, and withstand adversarial attempts. Assess the overall effectiveness in preventing unintended system interactions while maintaining model utility.

   2. Isolation Strategy Assessment: Assess the effectiveness of isolation strategies based on appropriateness for model capabilities, defense-in-depth implementation, consistency across model types, coverage of execution environments, alignment with industry practices, and evolution based on emerging risks. Evaluate whether isolation approaches are proportionate to the capabilities and potential risks of each model type.

   3. Documentation and Process Adequacy: Evaluate the quality of sandbox-related documentation, including clarity of security architecture, completeness of security controls for plugins, definition of capability boundaries, and security requirements for developers. Assess whether documentation is maintained as model capabilities evolve and expand.

   4. Continuous Improvement Mechanisms: Evaluate processes for improving sandbox security through regular testing of model boundaries, incorporation of lessons learned, adaptation to new capabilities, security architecture reviews, integration of emerging techniques, and red-teaming programs. Assess whether the organization demonstrates a commitment to continuously enhancing sandboxing controls as model capabilities and attack vectors evolve.

AIS-14: AI Cache Protection

Control Specification

Implement security measures to protect caches in GenAI systems and services.

Auditing Guidelines for Model Providers (MP)

Focus: The AI Model Provider has implemented effective security measures to protect caches in their model serving infrastructure and inference services, ensuring both performance optimization and protection of sensitive prompts, inference results, and embedding data.

1. Inquiry with Control Owners

   1. Interview Model Serving and Security Leadership: Interview ML infrastructure engineers, model serving specialists, and security architects responsible for foundation model deployment and cache implementation. Obtain and review the organization’s caching strategy and security policies covering inference result caching, prompt deduplication stores, embedding caches, token prediction optimizations, and model weight distribution systems. Verify documented security requirements exist for request isolation in cached content, cross-customer privacy in shared caches, KV-cache security during inference, and handling of potentially sensitive prompts in performance optimization systems.

   2. Review Caching Implementation Details: Examine documentation describing the technical implementation of caching within the model serving infrastructure, including inference result caching, token prediction optimization, model weight sharding and distribution, attention cache mechanisms, embedding vector stores, and token lookup tables. Assess how customer request isolation is maintained in high-throughput inference caches, how prompt data is protected, and how key-value cache mechanisms are secured during multi-turn conversations or batch inference operations.

   3. Assess Inference Privacy in Cached Content: Review mechanisms implementing privacy for cached inference operations and model interactions, including prompt isolation, inference result segregation, embedding vector privacy, and cache partitioning strategies. Evaluate how conversation history, multi-turn inference state, and token-level prediction caches are secured to prevent data leakage between customers while maintaining inference performance at scale.

   4. Evaluate Model Weight and Embedding Cache Security: Review procedures for model weight distribution and embedding cache management, including cache integrity verification, secure update mechanisms for model weight shards, access controls to cached embedding vectors, and protection of quantized weight caches. Assess monitoring systems for cache utilization patterns, anomalous access detection, potential inference reconstruction attacks, and signs of cache poisoning within the model serving infrastructure.

2. Obtaining and Verifying the Population of Records

   1. Define the Complete Population of Cache Components: Obtain a comprehensive inventory of caching mechanisms within the model serving infrastructure, including inference result caches, attention key-value caches, model weight distribution systems, token prediction optimizations, embedding vector stores, vocabulary lookup caches, batch inference optimization caches, and fine-tuning intermediate result stores. Include high-performance specialized caches, distributed model weight systems, and inference optimization mechanisms across the model serving architecture.

   2. Verify Population Completeness: Cross-reference the cache inventory against model serving architecture documentation, inference optimization strategies, deployment configurations, performance benchmark data, and service scaling documentation. Ensure the inventory aligns with model inference APIs, batch processing systems, fine-tuning infrastructure, and embedding generation services to confirm completeness of cache component identification across the model provider infrastructure.

   3. Categorize Cache Components by Risk Level: Segment the cache component population based on exposure to customer prompts, inference result accessibility, cache persistence duration, shared resource usage patterns, throughput requirements, model criticality, and potential privacy impact if compromised. This risk-based categorization should guide the depth and frequency of security assessment for each model-serving cache component.

3. Inspection of Evidence

   1. Cache Implementation Security Review: Select a representative sample of model-serving caching mechanisms based on risk levels, and verify the implementation of security controls, request isolation measures, and access restrictions. Examine cache configurations in inference systems, embedding services, and model weight distribution layers. Verify encryption of sensitive prompt data, implementation of request isolation between customers, and access control enforcement for cached inference artifacts.

   2. Customer Request Isolation Assessment: Review evidence of request isolation measures for cached content, including namespace separation, customer-specific encryption or tokenization, access boundary enforcement, and cache partitioning strategies. Evaluate how inference requests and results are protected from unauthorized access by other customers, how embedding caches maintain query isolation, and how attention mechanisms protect key-value caches during high-volume inference operations.

   3. Cache Invalidation and Model Update Controls: Verify implementation of cache invalidation mechanisms triggered by model updates, weight modifications, embedding algorithm changes, and security policy updates. Assess model weight distribution consistency strategies, version-tagged cache entries for model components, and coordination of cache refreshes across serving infrastructure to maintain consistent inference behavior during model updates.

   4. Cache Access Controls and Monitoring: Assess controls for cache access including authentication requirements for accessing model weights, authorization verification before serving cached inference results, service identity validation for internal component access, and customer context validation. Evaluate monitoring systems for cache access patterns, detection of unexpected access patterns, and audit logging of sensitive cache operations within the model serving platform.

   5. Protection Against Inference-Specific Cache Attacks: Examine the implementation of protections against cache poisoning, side-channel attacks through cache timing analysis, prompt reconstruction from caches, and inference result leakage. Assess input validation before caching inference results, output filtering of cached responses, protection against cache probing attacks, and defense mechanisms for shared inference cache infrastructure used across customer requests.

   6. Model Weight and Parameter Security: Review specialized protections for model weight caches and parameter stores, including integrity verification of cached weights, secure distribution mechanisms, tamper protection for model parameters, and access control enforcement. Assess how quantized model weights are protected in distributed serving environments and how weight caches are updated securely during model releases.

   7. Embedding and Vector Cache Protection: Verify the security of embedding and vector cache systems, including isolation between customer vector spaces, access controls for embedding lookups, and protection against embedding cache poisoning. Evaluate how nearest-neighbor caches and similarity search optimizations maintain data boundaries between customers while enabling high-performance vector operations.

4. Evaluation and Reporting

   1. Cache Security Effectiveness Assessment: Evaluate how well cache security implementations protect customer prompts and inference results while maintaining model serving performance. Assess the balance between caching for inference efficiency and appropriate security controls based on data sensitivity. Evaluate the effectiveness of defenses against unauthorized access, inference data leakage between customers, and cache-targeted attacks across the model serving infrastructure.

   2. Request Isolation Strategy Assessment: Assess the effectiveness of cache isolation strategies based on model serving architecture, inference patterns, prompt sensitivity, and performance requirements at scale. Evaluate whether security controls provide appropriate isolation between customer requests and whether defense-in-depth is implemented for the most sensitive cached inference components.

   3. Documentation and Process Adequacy: Evaluate the quality of cache-related security documentation, including clarity of caching architecture within the model serving platform, completeness of security controls, cache data lifecycle management, and incident response procedures. Assess whether documentation is maintained as model inference caching strategies evolve to support new capabilities and performance requirements.

   4. Continuous Improvement Mechanisms: Evaluate processes for improving cache security through regular security testing, incorporation of lessons learned from incidents, adaptation to new inference optimization techniques, security architecture reviews, and vulnerability management. Assess whether the organization demonstrates a commitment to continuously enhancing cache protection as model serving scales to more customers and new attack vectors emerge.

AIS-15: Prompt Differentation

Control Specification

Implement mechanisms enabling the model to clearly distinguish user-provided input instructions from data and system instructions (e.g., system prompts).

Auditing Guidelines for Model Providers (MP)

Focus: The AI Model Provider has implemented effective mechanisms enabling their foundation models to clearly distinguish user-provided input instructions from data and system instructions, ensuring consistent behavior and preventing prompt injection vulnerabilities.

1. Inquiry with Control Owners

   1. Interview Model Development and Safety Leadership: Interview research scientists, ML engineers, and safety specialists responsible for model training, alignment, and inference security. Obtain and review the organization’s approach to instruction separation, including: model architecture designs supporting instruction boundaries, training data formatting for instruction separation, instruction tuning methodologies, inference-time instruction handling, input formatting specifications and APIs, RLHF or other alignment techniques for instruction understanding. Verify documented approaches exist for enabling models to distinguish between user inputs and system instructions through techniques such as special tokens, role-based formatting, or architectural mechanisms within the model design itself.

   2. Review Model Training and Tokenization Implementation: Examine documentation describing the model’s instruction separation approach, including: tokenization strategy for instruction boundaries, special token design for role separation, training data formatting conventions, instruction-tuning dataset preparation, evaluation methods for instruction boundary understanding, and fine-tuning guidance for maintaining boundaries. Assess how the model’s training process and tokenization approach establish the foundation for reliable instruction separation during inference.

   3. Assess API and Interface Design: Review mechanisms implementing instruction separation at the interface level, including: API message structure designs, role-based input formatting (system/user/assistant), documentation of instruction boundary protocols, client library implementation of separation protocols, parameter validation and enforcement, and model-specific formatting conventions. Evaluate how the model provider’s interfaces enforce and maintain instruction boundaries for downstream users and applications.

   4. Evaluate Safety Monitoring and Testing: Review how the provider assesses and validates instruction separation through: red-team testing for instruction boundary violations, adversarial testing of instruction understanding, evaluation datasets for boundary maintenance, benchmark performance on instruction following, safety metrics related to instruction separation, and monitoring of customer usage patterns.

2. Obtaining and Verifying the Population of Records

   1. Define the Complete Population of Model Offerings: Obtain a comprehensive inventory of foundation models and inference interfaces, including: base language models of various sizes, fine-tuned and instruction-tuned variants, multimodal models with text components, embedding models with instruction handling, API endpoints and inference services, model weights for downloadable models, SDKs and client libraries.

   2. Verify Population Completeness: Cross-reference the model inventory against: public model documentation and specifications, published research papers and technical reports, API documentation and guidelines, model release notes and announcements, training methodology documentation, fine-tuning guidance for customers, and service capability descriptions. Ensure the inventory covers all models where instruction handling is relevant and instruction boundaries must be maintained.

   3. Categorize Models by Risk Level: Segment the model inventory based on: model capability and power, deployment method (API vs. downloadable), instruction complexity handling, customer exposure and usage volume, known sensitivities to prompt injection, fine-tuning availability, special capability access (plugins, tools). This risk-based categorization should guide assessment depth for each model.

3. Inspection of Evidence

   1. Model Instruction Separation Implementation Review: Select a representative sample of models based on risk levels and verify. For Architecture-Level Separation Mechanisms, examine how the model architecture supports instruction separation through techniques such as: special token embedding designs, attention mechanism adaptations for instruction awareness, architectural components supporting role distinction, position embeddings for context awareness, tokenizer design features for boundary detection. For Training-Level Separation Implementation, verify that models are trained to understand boundaries: training data formatting that clearly delineates roles, consistency of instruction formatting in training data, instruction fine-tuning methodology, reinforcement learning feedback for boundary violations, evaluation methods during training for boundary adherence. For Tokenization and Encoding Approach, confirm robust tokenization supporting separation: special tokens for role delineation, consistent tokenization of boundary markers, reserved token sequences for system instructions, handling of adversarial token sequences, and vocabulary design supporting instruction boundaries.

   2. API and Interface Assessment: Review how the model provider’s interfaces enforce separation. For Message Format Implementation, verify that APIs enforce structured message formats: distinct fields for system vs. user content, role-based message structures, validation of content against role expectations, proper handling of malformed requests, and documented formatting requirements. For Client Library Analysis, examine client libraries and SDKs to confirm: implementation of provider-recommended formats, helper functions encouraging proper separation, validation of inputs before transmission, clear documentation of boundary requirements, and consistent implementation across language bindings.

   3. Security Testing for Separation Effectiveness: Perform targeted testing of instruction boundary mechanisms. For Boundary Violation Testing, attempt to bypass instruction boundaries through: injecting system-like instructions in user content, mimicking boundary tokens within content, creating ambiguous role instructions, using token-level manipulation techniques, and attempting to override system instructions with user content. For Model Behavior Analysis, evaluate model responses to assess: consistent prioritization of system instructions, resistance to user-provided boundary confusion, appropriate handling of ambiguous input, maintenance of safety guardrails despite manipulation attempts, and generalization of boundary understanding to novel inputs.

   4. Documentation and Customer Guidance: Review supporting materials for API consumers. For API Documentation, verify existence and quality of: clear explanation of message formatting requirements, examples of proper instruction separation, warnings about improper formatting, best practices for boundary implementation, and guidance on recognizing and preventing injection. For Model Cards and Technical Reports, assess model documentation for disclosure of: instruction separation capabilities, known limitations in boundary enforcement, training approaches for boundary understanding, evaluation results on boundary maintenance, and security considerations for implementers.

   5. RLHF and Alignment Assessment: Evaluate alignment techniques supporting instruction boundaries. For Reinforcement Learning Implementation, verify that reinforcement signals address boundaries: reward design for proper instruction following, penalty signals for boundary violations, human feedback specifically targeting separation, alignment metrics for instruction understanding, iterative improvement process for boundary clarity. For Evaluation Suite Analysis, review evaluation benchmarks for: specific tests of instruction boundary maintenance, red-team evaluations of boundary violations, comparative performance across model versions, edge case handling in instruction scenarios, and generalization to novel boundary challenges.

4. Evaluation and Reporting

   1. Separation Effectiveness Assessment: Evaluate how well the implementation: enables models to consistently distinguish instruction types, maintains instruction priorities appropriate to their sources, resists manipulation attempts across varied inputs, balances flexibility with boundary enforcement, and addresses nuanced instruction understanding requirements.

   2. Architectural Strategy Assessment: Assess the effectiveness of the overall approach based on: integration of boundaries within model architecture, training methodology support for boundaries, interface design reinforcing separation, defense-in-depth implementation, and evolution across model generations.

   3. Documentation and Guidance Adequacy: Evaluate the quality of instruction separation documentation, including: clarity of API formatting requirements, completeness of boundary implementation guidance, transparency about limitations and edge cases, support for developers implementing interfaces, and ongoing communication about security considerations.

   4. Continuous Improvement Mechanisms: Evaluate processes for enhancing instruction boundaries through: regular red-team testing of boundary understanding, analysis of customer usage patterns and challenges, integration of lessons from boundary violations, research into improved architectural approaches, and iterative enhancement of training methodologies.

BCR: Business Continuity Management and Operational Resilience

BCR-01: Business Continuity Management Policy and Procedures

Control Specification

Establish, document, approve, communicate, apply, evaluate and maintain business continuity management and operational resilience policies and procedures. Review and update the policies and procedures at least annually, or upon significant changes.

Auditing Guidelines for Model Providers (MP)

1. Verify that the BCM policies cover the continuity of the model lifecycle, specifically addressing disruptions to model training and retraining infrastructure (e.g., compute, GPU availability), the regular backup of model versions, and the establishment of fallback procedures for AI service outages. This should include provisions for model version rollback and fallbacks in case deployments fail or are corrupted, and handling adversarial events like model poisoning.

2. Inspect evidence of documented Business Continuity Plans (BCPs) for all stages of the model lifecycle, including procedures for restoring model versions and maintaining operational continuity during adversarial events (e.g., model misuse, dataset loss).

3. Check the last update date of these procedures and inspect documentation of test runs (e.g., model restoration drills, failover to a fallback model) to ensure the continuity and resilience of AI services.

4. Verify that all policies and procedures are formally reviewed at least annually or upon significant changes, with updates documented through version history and approvals, and communicated to relevant stakeholders.

BCR-02: Risk Assessment and Impact Analysis

Control Specification

Determine the impact of business disruptions and risks to establish criteria for developing business continuity and operational resilience strategies and capabilities. Review and update the risk assessment and impact analysis at least annually, or upon significant changes.

Auditing Guidelines for Model Providers (MP)

1. Obtain and Review the BIA/Risk Assessment: Confirm it includes risks such as data poisoning, model drift, adversarial attack scenarios, service unavailability, and training/inference downtime. Look for RTO/RPO definitions specific to model retraining, rollback, or replacement.

2. Verify Inclusion of Model Lifecycle Events: Ensure assessments cover disruptions across data sourcing, training infrastructure, deployment, and access APIs.

3. Check Review and Update Procedures: Confirm that risk and BIA reviews are conducted at least annually or after significant model version changes, architecture shifts, or policy updates.

4. Inspect Evidence: Meeting minutes, updated BIA logs, documented threats, or incident simulations.

BCR-03: Business Continuity Strategy

Control Specification

Establish strategies to reduce the impact of business disruptions, and improve resiliency and recovery from business disruptions.

Auditing Guidelines for Model Providers (MP)

1. Confirm the business continuity strategy includes secure backup and version control of AI models.

2. Verify procedures exist to quickly redeploy validated models in the event of infrastructure or storage failures.

3. Check that critical ML pipelines and retraining mechanisms are resilient to disruptions.

4. Ensure dependencies on cloud GPU/TPU resources are mapped and mitigated through alternative options.

5. Validate coordination with OSP to recover model inference services in multi-tenant environments.

6. Confirm disaster recovery strategies include protection for sensitive training datasets and model artifacts.

BCR-04: Business Continuity Planning

Control Specification

Establish, document, approve, communicate, apply, evaluate and maintain a business continuity plan based on the results of the operational resilience strategies and capabilities.

Auditing Guidelines for Model Providers (MP)

1. Confirm the BCP includes recovery procedures for training environments, model registries, and model artifact storage.

2. Verify strategies exist for restoring access to pre-trained models and ongoing model training during outages.

3. Ensure continuity plans address interruptions to hyperparameter tuning platforms or experiment tracking systems.

4. Check contingency measures for loss or corruption of high-value model artifacts (e.g., versioned checkpoints).

5. Validate simulation or drill evidence showing timely recovery of critical model development environments.

6. Ensure communication plans exist to notify OSP/AP of model availability or retraining disruptions.

7. Verify the BCP includes mitigation strategies for supplier-related disruptions (e.g., training data vendors, compute providers).

BCR-05: Documentation

Control Specification

Develop, identify, and acquire documentation, both internally and from external parties, that is relevant to support the business continuity and operational resilience plans. Make the documentation available to authorized stakeholders and review at least annually, or upon significant changes.

Auditing Guidelines for Model Providers (MP)

1. Confirm the existence of documentation for business continuity plan, operational resilience plan, and recovery procedures related to model training pipelines, model registries, and experiment tracking tools.

2. Verify that records exist for dependencies on data lakes, feature stores, and cloud training clusters.

3. Ensure version-controlled documentation is maintained for critical model artifacts and retraining schedules.

4. Validate inclusion of upstream and downstream documentation links (e.g., training data providers, application consumers).

5. Check documentation for supplier agreements, model licensing, and data usage contracts that affect continuity.

6. Confirm documentation includes disaster recovery test results and timelines for model infrastructure.

7. Ensure that documentation review and approval logs are available for each update cycle or major architecture shift.

BCR-06: Business Continuity Exercises

Control Specification

Exercise and test business continuity and operational resilience plans at least annually, or upon significant changes.

Auditing Guidelines for Model Providers (MP)

1. Examine the plans for business continuity and operational resilience tests regarding their intended outputs.

2. Examine the schedules of such tests and their periodicity.

3. Evaluate if the plans are tested upon significant changes or at least annually. Verify the existence of documented test results, inspect the test results, confirm who has performed the test, and confirm that the test is taking place annually. Confirm the successful/unsuccessful results of the test and look for lessons learned.

4. Verify that exercise scenarios include failure modes specific to model serving infrastructure, model availability issues, and recovery procedures for model artifacts and metadata.

5. Review exercise results to confirm that model recovery procedures were successfully executed, including validation that recovered models maintain expected performance, accuracy, and security characteristics.

6. Assess whether exercises included verifying model version control during recovery scenarios and testing the integrity of model assets after restoration.

7. Examine evidence that exercises tested the capability to serve models from alternate infrastructure when primary systems are unavailable.

8. Verify that exercise planning, execution, and results were reviewed by appropriate ML engineering and operations leadership and that identified issues were addressed through documented improvement plans.

BCR-07: Communication

Control Specification

Establish and maintain communication channels with all relevant stakeholders in the course of business continuity and resilience procedures.

Auditing Guidelines for Model Providers (MP)

1. Examine the business continuity and operational resilience plans for determining stakeholders.

2. Determine if the organization has identified stakeholders.

3. Examine the procedures for communication with identified stakeholders.

4. Verify that a communications plan exists. The plan should include up-to-date contacts, clear roles and responsibilities, a period for updates, and escalation procedures.

5. Verify that model status notification systems can communicate availability, performance issues, or version impacts to dependent services.

6. Review evidence that the MP has established methods to provide estimated resolution timeframes and regular updates during model disruption incidents.

7. Assess the MP’s procedures for communicating model-specific technical details to technical stakeholders while providing appropriate simplified information to non-technical audiences.

8. Verify that communication protocols exist for coordinating with infrastructure providers during incidents where model issues may stem from underlying infrastructure problems.

9. Review records from past model disruption incidents or exercises to confirm effective communication with stakeholders, including clarity about model versions affected and workaround options.

10. Confirm that the MP has established methods to verify receipt of critical communications by key stakeholders and escalation procedures for unacknowledged notifications.

BCR-08: Backup

Control Specification

Periodically perform backups. Ensure the confidentiality, integrity and availability of the backup, and verify restoration from backup for resiliency.

Auditing Guidelines for Model Providers (MP)

1. Examine the data classification policy for identifying data that requires a backup.

2. Examine the requirements for the security of such backups.

3. Evaluate the effectiveness of the backup and restore.

4. Determine if backup and restore procedures are tested periodically.

5. Examine if the backup and restore procedures accomplish the organization’s SLAs.

6. Evaluate the testing backup restorations, ensuring recovery time objectives (RTO) and recovery point objectives (RPO) are met.

7. Verify implementation of model versioning systems that maintain consistent backups of model artifacts throughout the model lifecycle, including development and production versions.

8. Assess encryption and access controls protecting the confidentiality of backed-up training data and model artifacts, particularly for sensitive or proprietary models.

9. Review backup validation procedures that verify the integrity of backed-up model components through checksums, validation tests, or other verification mechanisms.

10. Examine documentation and test results demonstrating successful model restoration, verifying that restored models maintain expected performance metrics and inference capabilities.

11. Verify that backup systems capture sufficient metadata and configuration parameters to enable model reproducibility or retraining if source data or code is lost.

12. Assess backup strategies for large training datasets, including differential or incremental backup approaches and deduplication methods to manage storage requirements.

BCR-09: Disaster Response Plan

Control Specification

Establish, document, approve, communicate, apply, evaluate and maintain a disaster response plan to recover from natural and man-made disasters. Update the plan at least annually, or upon significant changes.

Auditing Guidelines for Model Providers (MP)

1. Examine the disaster recovery plan and procedures for adequacy, approval, communication, and effectiveness as applicable to a disaster response plan.

2. Examine the disaster recovery plan and procedures for evidence of review upon significant changes or at least annually.

3. Determine if the disaster recovery plan has periodic updates and ensure that all personnel are regularly trained on it.

4. Verify that the plan has received formal approval from leadership responsible for model development and operations, with evidence of review and sign-off.

5. Assess the defined recovery priorities for different models and model versions based on their criticality to dependent services.

6. Review documentation of model artifact backup and recovery procedures, including verification of model reproducibility after environment recovery.

7. Verify that geographically distributed copies of critical model artifacts and training data are maintained to support recovery after regional disasters.

8. Examine evidence that the disaster response plan has been communicated to all relevant model development and operations personnel, including training records.

9. Review records of model recovery tests conducted within the past 12 months, confirming they validated successful restoration of model serving capabilities.

10. Verify that the plan is reviewed and updated at least annually and after significant changes to model architectures or serving infrastructure, with documented change history.

BCR-10: Response Plan Exercise

Control Specification

Exercise the disaster response plan annually or upon significant changes, including, if possible, the participation of local emergency authorities.

Auditing Guidelines for Model Providers (MP)

1. Determine if the organization has identified local emergency authorities contacts and, if possible, has included them in the disaster recovery plan exercise.

2. Examine the organization’s policies for planning and scheduling disaster response exercises and, if possible, involving local emergency authorities.

3. Evaluate if plans are tested upon significant changes or at least annually.

4. Determine if the organization has a feedback mechanism post-exercise to ensure lessons learned are incorporated into future exercises.

5. Verify that exercises tested the recovery of different model types and versions based on their criticality to dependent services.

6. Review exercise documentation to confirm that model performance and accuracy were validated after completing recovery procedures.

7. Assess whether the exercises tested recovery from various failure scenarios, including infrastructure unavailability, data corruption, or model-serving platform failures.

8. Verify that model version consistency was checked across recovery scenarios to restore the correct models.

9. Confirm that exercises included coordination with infrastructure providers to simulate realistic recovery sequences and dependencies.

10. Review documentation of lessons learned from exercises and verify that identified weaknesses in model recovery capabilities resulted in documented improvement plans with clear ownership and timelines.

11. Verify that additional exercises were conducted following significant changes to model architectures, serving infrastructure, or recovery procedures.

BCR-11: Equipment Redundancy

Control Specification

Supplement business-critical equipment with both locally redundant and geographically dispersed equipment located at a reasonable minimum distance in accordance with applicable industry standards.

Auditing Guidelines for Model Providers (MP)

1. Examine the process to identify business-critical equipment and any redundant equipment.

2. Examine the process to identify applicable industry standards.

3. Evaluate if the redundant business-critical equipment is independently located at a reasonable distance.

4. Verify that redundant model-serving equipment is deployed across geographically separated locations, following relevant industry standards and recovery time objectives.

5. Review implementation of redundant model storage systems, including replication mechanisms that ensure model artifacts and training data are duplicated across separate locations.

6. Assess automated model deployment capabilities across redundant environments, verifying models can be consistently deployed to alternative equipment when primary systems fail.

7. Verify monitoring systems that detect performance degradation or failures in model serving equipment and trigger appropriate failover responses.

8. Examine load balancing mechanisms that distribute model inference requests across redundant processing equipment to prevent overloading single systems.

9. Review records of failover tests conducted between redundant model-serving environments, confirming that models maintain performance characteristics when transitioning between environments.

10. Verify that critical model development environments, where applicable to operations, also have appropriate equipment redundancy to prevent disruption to model improvement workflows.

CCC: Change Control and Configuration Management

CCC-01: Change Management Policy and Procedures

Control Specification

Establish, document, approve, communicate, apply, evaluate and maintain policies and procedures for managing the risks associated with applying changes to assets owned, controlled or used by the organization. Review and update the policies and procedures at least annually, or upon significant changes.

Auditing Guidelines for Model Providers (MP)

1. Conduct interviews with personnel responsible for documenting, maintaining, and communicating organizational change management policies, procedures, and standards (the Policies).

2. Inspecting Records and Documents: Obtain and review the change management Policies to ensure they are adequate for the organization to manage risks associated with applying changes to organizational assets. Verify that the Policies define the personnel or roles responsible for their dissemination, identify an official accountable for managing the Policies, specify the frequency of reviews and updates (annually), and outline events that necessitate policy updates.

3. Verify that the Policies are disseminated, are reviewed and updated at least annually or upon significant changes, are approved, and are communicated to relevant stakeholders.

CCC-02: Quality Testing

Control Specification

Establish, maintain and implement a defined quality change control, approval and testing process incorporating baselines, testing, and release standards.

Auditing Guidelines for Model Providers (MP)

1. Interview Key Stakeholders and Review Documentation

   1. Model Development and Operations: Understand model architecture design and versioning strategies. Examine training data curation, filtering, and dataset versioning. Review fine-tuning methods, weight management, and training infrastructure configurations. Assess reproducibility controls (e.g., checkpoints, orchestration, naming conventions). Confirm transition processes from research to production stages.

   2. Review model evaluation frameworks and benchmark testing protocols. Examine safety and red-teaming practices, fairness and bias assessments, and transparency standards (e.g., model cards). Confirm deployment qualification criteria and performance verification methods. Validate governance mechanisms such as model review boards and promotion approval workflows.

   3. Assess training data sourcing, documentation, filtering, and contamination controls. Review dataset version tracking, test/train/validation split practices, and impact assessments for data changes.

   4. Confirm baseline practices ensure: release consistency and safety, performance stability, reproducibility, and lifecycle governance.

2. Obtain and Verify Model Release Records

   1. Collect the full population of model release data from: model registries, training run databases, version control systems, tokenizer/config repositories, API deployment logs, model card publications, and cross-check against change management systems to confirm completeness and integrity of records.

3. Sample and Inspect Model Releases

   1. Sample Selection: Choose a representative set of releases, including: major architectural changes and incremental improvements, fine-tuned and domain-specialized variants, and safety-enhanced or emergency releases.

   2. For each sample, verify: Benchmark and task-specific performance, red-teaming, robustness, and harmful content evaluation, fairness and alignment checks, training reproducibility (e.g., seed, infra, dataset, parameters).

   3. Ensure releases were approved by: research and product leads, model governance and AI ethics boards, security, legal, and safety teams.

   4. Validate formal promotion stages were followed: Research → Alpha → Beta → Production, progressive exposure and rollback planning, and compatibility and version management practices.

   5. Check that documentation includes: accuracy, latency, throughput, resource and scaling metrics, known limitations, and uncertainty quantification.

   6. Verify reproducibility is supported through: hyperparameters and optimizer configurations, training infrastructure and random seeds, preprocessing and data split documentation.

   7. Evaluate the completeness of model cards, covering: intended use and limitations, performance and fairness across demographics, safety mitigations and environmental impact.

CCC-03: Change Management Technology

Control Specification

Implement a change management procedure to manage the risks associated with applying changes to assets owned, controlled or used by the organization.

Auditing Guidelines for Model Providers (MP)

1. Inquiring with Control Owners: Conduct interviews with owners of the MP’s change management process and review supporting documentation to understand how changes to systems, code, configurations, and AI/ML assets are governed. Focus on procedures for submitting change requests, conducting risk assessments, securing approvals, performing testing, planning implementation, and executing post-deployment verification. Ensure oversight of both internally managed and outsourced technology assets, and clarify how the MP uses the following systems to implement and monitor change management: change tracking platforms (e.g., ServiceNow, Jira Service Management), version control systems (e.g., GitHub, GitLab, Azure DevOps), ML model registries and lineage tools (e.g., MLflow, DVC, proprietary alternatives), automated testing frameworks (e.g., PyTest), and deployment validation tools.

2. Inspecting Records and Verifying Controls: Select a representative sample of change records from change tracking platforms, version control commits, model registry entries, and automated testing results. For each sampled change, confirm it followed documented processes for approval, risk analysis, testing, and validation. Verify integration and use of relevant change tooling (e.g., CI/CD pipelines, model registries). Ensure the organization applies the same control rigor across both internal and third-party managed environments.

CCC-04: Unauthorized Change Protection

Control Specification

Implement and enforce a procedure to authorize the addition, removal, update, and management of assets that are owned, controlled or used by the organization.

Auditing Guidelines for Model Providers (MP)

1. Inquiring with Control Owners

   1. Interview Model Development Leadership and Model Governance Teams: Interview AI research leaders, model development teams, and governance/safety teams to understand requirements. For Access Control and Role-Based Permissions: Access control frameworks for modifying model components including architecture design, training data curation, model weight checkpoints, hyperparameter configurations, and safety alignment mechanisms; Role-based access for architecture code modifications, training data access and filtering, hyperparameter changes, model weight management, safety mechanism adjustments, and evaluation benchmark modifications; Separation of duties between research vs. production engineering, architecture design vs. training operations, safety evaluation vs. performance optimization, and data curation vs. model deployment. For Model Development Infrastructure: model architecture design and implementation processes, training data curation and preprocessing pipelines, training hyperparameter configurations and model weight management, evaluation suite and benchmark systems, model distribution and versioning infrastructure, and distribution API configurations. For Governance and Approval Workflows: Approval workflows for new model architecture deployments, major training dataset modifications, model weight updates and releases, and safety alignment mechanism changes; Public release qualification processes and model deprecation with successor transitions; Model lifecycle governance including safety and red-teaming evaluations, model alignment assessments, and bias and fairness validations; Release readiness reviews, legal and compliance evaluations, and customer impact assessments.

   2. Review Change Management and Access Control Documentation: Examine documentation describing model change management policies, deployment controls, and authorization requirements. For Change Management and Deployment Controls: Component-specific change categories and approval requirements with emergency change procedures for safety-critical components; Model versioning and release policies with training data version control requirements; Testing standards for model changes and approval gates for model releases; Validation checks for model performance and safety evaluation requirements; Progressive deployment strategies and monitoring requirements for new model versions. For Feature and Capability Management: Capability access control processes and emergency capability restriction procedures; API behavior versioning approaches and customer access tier management; Model capability audit procedures. For Access Control Authorization Requirements: Model architecture code repositories and training data access with pipeline configurations; Distributed training infrastructure and model weight artifacts with checkpoints; Safety alignment mechanisms and model evaluation systems; Distribution API configuration and customer access management systems.

2. Obtaining and Verifying the Population of Records

   1. Obtain the complete population of AI models under review, including production models, fine-tuned variants, and models in development. Validate completeness by reviewing model registry databases, version control systems for model weights, dataset access logs, training pipelines, and deployment logs from platforms like HuggingFace, internal model hubs, or commercial model provider interfaces.

3. Inspecting Records and Documents: Select a representative sample of AI models and obtain the complete list of data scientists, ML engineers, and systems with permission to create/modify/fine-tune models, training parameters, or deployment configurations. Verify that access to alter models of their components is properly restricted by examining role-based access settings and access logs, reviewing approval workflows for model changes, and testing that unauthorized personnel cannot initiate model retraining or parameter modifications.

   1. Select Representative Sample: Choose a balanced sample of models based on: different model types and capabilities, various scales (parameter counts), range of deployment stages, different maturity levels and release status, mix of safety alignment approaches, varying risk categorizations, and different access tiers and availability.

   2. Obtain Access Control Information: For each sampled model, collect the complete list of personnel and systems with modification capabilities: Research and Development Personnel (research scientists, model architects, ML engineers, data scientists, safety researchers, training operations engineers, evaluation engineers), Administrative Personnel (model governance committee members, release managers, safety evaluation teams, quality assurance specialists, API and product managers), and Automated Systems (training pipelines and infrastructure, evaluation and benchmarking systems, deployment automation systems, model registry and artifact management safety evaluation frameworks, API configuration management systems).

   3. Validate Access List Completeness: Verify the completeness of access lists through: reviewing script logic for access report generation, cross-referencing with identity management systems, comparing against role definition repositories, validating against authentication logs, reconciling with customer subscription records, and examining API gateway access configurations.

   4. Verify Access Restrictions and Model-Specific Access Controls: For each sampled model, confirm that access to modify the model and its components is properly restricted. For Repository and Code Controls: Branch protection for model architecture code with code review requirements for critical components; Access limitations for training configurations and commit signing requirements for safety mechanisms; Repository access audit logs and architecture change approval processes; Component library access management and architecture experimentation controls. For Training Pipeline and Data Controls: Approval requirements for training runs with separation of duties in training operations; Dataset access controls and permissions with data filtering configuration changes; Training infrastructure access restrictions and data preprocessing pipeline modifications; Training launch approval audit trails and data quality monitoring access. For Model Artifact and Weight Management: Access restrictions for model weights and checkpoints with approval workflows for promoting model artifacts; Audit logging for model artifact access and integrity verification mechanisms; Model registry access controls and weight versioning with tagging access; Checkpoint promotion permissions and quantization with optimization controls. For Safety and Distribution Controls: Safety alignment code access and safety parameter configuration changes; Safety evaluation access controls and safety override authorization; API configuration access and model serving infrastructure changes; Version availability controls and rate limiting with quota configuration. For Approval Workflow Validation: Required approvals obtained for model changes with safety evaluations for appropriate changes; Performance benchmark approvals and compliance review documentation where required; Emergency change procedures and approvals.

CCC-05: Change Agreements

Control Specification

Include provisions limiting changes directly impacting service customers owned environments (tenants) to explicitly authorized requests within service level agreements.

Auditing Guidelines for Model Providers (MP)

1. Inquiring with Control Owners

   1. Conduct interviews with personnel responsible for managing AI model updates and customer model deployment changes to understand authorization processes for modifying model behavior or performance in customer implementations. Verify their understanding of controls that prevent unauthorized changes to model outputs, API responses, or inference behavior that directly impact customer applications and use cases.

2. Inspecting Records and Documents

   1. Review AI Model Deployment Change Policies: Evaluate policies governing model version updates, fine-tuning modifications, API behavior changes, and inference parameter adjustments that affect customer model implementations.

   2. Inspect Customer Model Licensing or API Agreements: Look for restrictions on automatic model updates, changes to output behavior, API response modifications, or alterations to model performance characteristics.

   3. Assess Model Version Control or Rollback Mechanisms: Customers should be able to maintain specific model versions or reject performance-impacting updates. Review model versioning support, API version controls, or customer-specified model configurations.

   4. Verify Model Change Authorization Processes: Examine documented procedures requiring explicit customer authorization before implementing changes to model behavior, API functionality, or inference characteristics that directly impact customer applications.

   5. Review Customer Model Update Documentation: Validate that customers receive proper notification and authorization requests before model changes that affect output quality, response time, or functional behavior.

   6. Examine SLA Compliance for Model Performance Modifications: Confirm that model updates maintain agreed performance parameters and customer-authorized functional specifications.

   7. Review API and Model Change Disclosure Policies: MP-initiated changes (e.g., model behavior, latency, input schema) may impact customer systems. Check documentation or support policies outlining notice requirements.

   8. Inspect Customer-Facing Agreements or Versioning Guarantees: Customers may rely on specific model behavior for regulated or critical functions. Review SLAs or ToS for clauses limiting backward-incompatible changes.

   9. Confirm Change Approval and Communication Protocols: Changes should be disclosed and authorized if affecting contracted use cases. Check if change management systems log customer notifications or consent capture.

CCC-06: Change Management Baseline

Control Specification

Establish, document and implement change management and configuration baselines for all relevant authorized changes on organization assets. Review and update the baselines at least annually, or upon significant changes.

Auditing Guidelines for Model Providers (MP)

1. Verify documented baselines exist and define what is “in baseline” for: model architecture/code, training data composition + preprocessing pipelines, training configurations/hyperparameters, model artifacts (weights/checkpoints, tokenizers/vocab, quantized/distilled variants), evaluation benchmarks/metrics + acceptance thresholds (incl. safety/alignment/robustness), and distribution/runtime configurations (e.g., API parameters, constraints, versioning).

2. Confirm a change process exists and is followed for all relevant authorized baseline changes (request - approval - version control - validation - release), including role-based authorization/segregation for high-impact changes (e.g., data, weights, safety filters, distribution settings).

3. Select a sample of recent and/or higher-risk model changes (e.g., new release, re-train/fine-tune, tokenizer update, safety filter change) and verify end-to-end traceability: approved change record, linked versions (code/data/config/artifacts), evidence of reproducibility/traceability for the training run, evaluation results vs prior version (incl. safety/alignment), and release/distribution updates consistent with approval.

4. Verify controls protecting baseline integrity for sensitive artifacts (e.g., weights/checkpoints, tokenizers): access controls, approval gates, integrity checks/chain-of-custody as applicable, and audit trails for modifications.

5. Confirm baselines are reviewed/updated at least annually and upon significant changes (e.g., major architecture change, material training data shift, new training framework/environment, critical vulnerability, major distribution/runtime change), with evidence of the review and resulting updates where applicable.

CCC-07: Detection of Baseline Deviation

Control Specification

Implement detection measures with proactive notification in case of changes deviating from the established baseline.

Auditing Guidelines for Model Providers (MP)

1. Inquiry with Control Owners

   1. Interview monitoring and operations personnel responsible for detecting changes to foundation models. Obtain and review the organization’s monitoring strategies, alert thresholds, and notification workflows for: model behavior consistency across versions, safety alignment mechanism effectiveness, inference performance and reliability, API and distribution system stability, training pipeline integrity, and weight integrity and artifact consistency. Verify the existence of documented detection mechanisms for: model behavioral drift from expected patterns, safety boundary violations or degradation, performance regression across benchmark tests, training reproducibility deviations, inference service anomalies, and unexpected deployment artifacts. Identify monitoring tools used for: continuous model evaluation on benchmark datasets, red-teaming and adversarial testing automation, inference performance and latency monitoring, model output distribution analysis, safety alignment verification systems, and model weight and artifact verification.

   2. Review Notification and Response Procedures: Examine documentation describing notification pathways when model issues are detected. Understand escalation procedures based on impact severity and model criticality. Verify integration between detection systems and research/engineering teams. Assess emergency response capabilities for high-severity model safety incidents. Review response playbooks for different types of model-related issues: safety alignment failures or degradation, performance regression on critical tasks, unexpected model behavior changes, API integration compatibility issues, inference reliability degradation, and customer-reported model issues.

2. Obtaining and Verifying the Population of Records

   1. Define the complete population of monitoring records by inventorying monitoring systems for foundation models, including model evaluation and benchmark systems, safety testing and alignment verification tools, performance monitoring across deployment environments, inference API reliability tracking, customer usage pattern analysis systems, training reproducibility verification tools, model artifact integrity verification systems, and adversarial and red-team testing frameworks.

   2. Verify completeness of the population by cross-referencing monitoring coverage against the inventory of foundation models, assessing completeness of alerting rules for enterprise customer impact scenarios, or verify that monitoring covers all deployment environments (dev, staging, production).

3. Inspection of Evidence

   1. Monitoring System Verification: Verify that monitoring systems are configured to detect deviations in the following categories. For Model Behavior Consistency: output distribution shifts from baseline behavior, reasoning capability degradation, task-specific performance regressions, significant style or tone variations, unexpected capability changes, and knowledge or factual accuracy shifts. For Safety Alignment Effectiveness: safety boundary compliance rates, harmful content generation incidents, adversarial prompt resilience, red-team testing pass/fail rates, bias metric shifts from baseline, and refusal behavior consistency. For Performance Characteristics: inference latency deviations, token generation speed changes, resource utilization patterns, error rate variations, context window handling consistency, and batch processing efficiency. For Distribution Infrastructure: API availability and reliability, request throttling effectiveness, load balancing performance, model serving correctness, deployment artifact integrity, and version consistency across regions.

   2. Alert Configuration Assessment: Examine alert configuration to verify: appropriate thresholds for different model capabilities, safety-critical vs. performance-related alert priorities, phased alerting based on deviation magnitude, context-rich alert payloads with examples, different thresholds for research vs. production models, integration with model version control systems, and correlation between related behavioral changes.

   3. Sample-Based Testing of Detection Capabilities: Select a representative sample of foundation models and perform controlled tests: introduce synthetic behavioral deviations, submit adversarial safety boundary tests, simulate performance degradation scenarios, create model artifact inconsistencies, test version mismatch detection, and generate unusual usage patterns. Verify that monitoring systems: accurately detect the simulated issues, generate appropriate alerts with correct severity classification, include representative examples in context, trigger within expected timeframes, follow defined notification workflows based on issue type, and correctly identify the affected model capabilities.

   4. Alert Notification Workflow Verification: Trace the notification path for different types of model issues: initial detection enrichment with examples, routing to appropriate teams (research, engineering, safety), escalation for critical safety or performance issues, customer communication preparation workflows, integration with version rollback systems, evidence collection for post-incident analysis, and coordination between safety and engineering teams.

   5. Response Effectiveness Evaluation: Review historical model-related incidents to evaluate: time to detect behavioral deviations, accuracy of issue characterization, response time to critical safety issues, effectiveness of mitigation actions, customer communication timeliness and transparency, root cause analysis thoroughness, and detection refinement following incidents.

   6. Automated Remediation Assessment: Verify implementation of automated remediation for common issues: automatic traffic shifting to stable model versions, dynamic safety filter adjustments, inference parameter optimization for performance, automated rollback triggers for critical issues, gradual rollout suspension on anomaly detection, and safety boundary reinforcement mechanisms.

   7. Integration with Research and Development: Assess how detection systems feed back into model development: integration of detected issues into training data, correlation of behavioral shifts with training changes, feedback loops for safety boundary improvements, detection-informed benchmark development, historical deviation analysis for research insights, and continuous improvement of detection sensitivity.

CCC-08: Exception Management

Control Specification

Implement a procedure for the management of exceptions, including emergencies, in the change and configuration process. Align the procedure with the requirements of GRC-04: Policy Exception Process.

Auditing Guidelines for Model Providers (MP)

1. Inquiry with Control Owners

   1. Interview AI Research Leadership and Review Exception Management Policies: Interview AI research leadership, model development teams, and safety researchers responsible for exception handling in foundation model development and deployment, and obtain organizational exception management policies including emergency model weight update procedures, critical safety alignment intervention processes, expedited model release procedures, model behavior emergency correction processes, model API service disruption handling, documentation requirements for exceptions, and post-exception review procedures, while verifying documented criteria for model safety emergencies requiring immediate action, critical model behavior issues requiring expedited changes, safety or ethical concerns requiring urgent intervention, model performance degradation requiring immediate attention, and proper authorization levels for different exception types.

   2. Review Exception Process Documentation and Emergency Response Procedures: Examine documentation describing exception request workflows and templates for model changes, required approvals based on exception type and downstream impact, risk assessment procedures for model exception requests, temporary approval timeframes for emergency model changes, documentation requirements for model change justification, post-implementation validation requirements for model releases, and exception tracking and reporting mechanisms for model versions, while reviewing procedures for handling safety alignment failure response, model behavior anomaly handling, harmful capability discovery response, model API performance degradation management, critical security vulnerability remediation, model weight corruption or integrity issue handling, data privacy or training data leakage response, and public relations crisis management for model issues.

   3. Evaluate Exception Governance Framework: Verify the existence of governance structures for model exception management including exception approval authorities and escalation paths, emergency response teams and on-call rotations for different model types, exception review committees and their charters, governance alignment with GRC-04 requirements, integration with enterprise risk management, and executive oversight of model exception patterns.

2. Obtaining and Verifying the Population of Records:

   1. Define the Complete Population of Exception Records: Obtain a comprehensive inventory of model change exceptions, including: emergency model weight or parameter updates, expedited safety alignment adjustments, unplanned model version rollbacks, emergency model capability restrictions, API service emergency modifications, training or fine-tuning emergency interventions, temporary model access limitation exceptions, and retroactive approvals for emergency model actions.

   2. Verify Population Completeness: Cross-reference exception records against: model behavior monitoring alert logs, safety testing failure reports, customer reporting of model issues, red team testing urgent findings, model API service incident logs, customer impact communications, post-incident review documentation, and risk acceptance registers for model capabilities.

3. Inspection of Evidence

   1. Exception Sample Selection: Select a representative sample of model exceptions based on: different types of exceptions (safety interventions, behavior corrections), various model types (language models, diffusion models, multimodal), range of impact levels (high, medium, low), different time periods to assess consistency, exceptions approved by different authorities, and various justification categories (safety concerns, performance issues).

   2. Exception Sample Testing: For each selected model exception, verify the following. For Proper Justification: clear documentation of the AI application exception need, evidence supporting the urgency or special circumstances (test results, customer reports), appropriate risk assessment including downstream impact, consideration of alternatives (e.g., capability restriction vs. full rollback), and alignment with exception criteria for model changes. For Appropriate Approval: approval by authorized individuals/committees (or retroactive approval for true emergencies), approval obtained at appropriate level for risk and impact, documentation of approval decision for model changes, and conditions or limitations clearly specified (e.g., timebound, specific models). For Appropriate Implementation: exception implemented as approved (verified through model logs), scope limited to what was approved (e.g., specific model versions), temporal limitations respected for temporary changes, additional monitoring implemented during exception period, mitigating controls properly applied (e.g., customer notifications), and communication to model consumers about changes or limitations. For Proper Closure and Follow-up: exception closed when no longer needed, post-exception validation performed (e.g., comprehensive testing), return to standard process confirmed, lessons learned documented in post-mortem, model development process improvements identified, and retroactive documentation completed for emergency changes.

   3. Exception Tracking and Reporting: Verify the effectiveness of model exception tracking mechanisms: centralized repository of all model exceptions, tracking of exception status and expiration for temporary changes, regular reporting to model governance bodies, trend analysis of model exception patterns, identification of recurring model behavior issues, integration with risk management reporting, and downstream impact tracking across exceptions.

   4. Exception Governance Assessment: Assess the effectiveness of model exception governance: appropriate escalation based on downstream impact and urgency, executive visibility into significant model exceptions, pattern recognition and systemic improvements for model issues, balancing of model quality/safety and emergency response, incorporation of lessons learned into model development, and integration with broader research governance frameworks.

   5. Continuous Improvement Mechanisms: Evaluate processes for improving model exception handling: regular review of model exception patterns, process refinements based on lessons learned from major incidents, reduction of recurring exceptions through training improvements, calibration of emergency response capabilities for different model failure modes, enhancement of risk assessment in time-sensitive model situations, evolution of exception criteria based on operational experience, and architectural improvements to reduce emergency change requirements.

CCC-09: Change Restoration

Control Specification

Define and implement a process to proactively roll back changes to a previous known good state in case of errors or security concerns.

Auditing Guidelines for Model Providers (MP)

1. Inquiry with Control Owners

   1. Interview model development leadership, ML engineers, safety teams, and deployment architects to understand how rollback capabilities are defined, implemented, and governed across the model lifecycle. Verify that the organization has clearly defined rollback criteria such as safety violations or misalignment incidents, performance degradation below defined thresholds, capability regressions or functional failures, security vulnerabilities affecting models, and customer-impact incidents. Confirm that authority matrices for rollback decisions are documented, including escalation paths for emergency scenarios. Review rollback policies for different model types and deployment environments. Evaluate the organization’s rollback planning processes, which should include: step-by-step rollback procedures for model weights, serving infrastructure, APIs, and tokenizer configurations, compatibility assurance across versions and components, testing protocols prior to rollback deployment, communication plans for model consumers, and documentation of rollback execution and lessons learned.

   2. Assess how the organization defines and manages known good states, including comprehensive evaluation and validation criteria; preservation of model artifacts, checkpoints, and parameters; safety alignment, bias/fairness benchmarks, and capability baselining; and tagging and versioning mechanisms to ensure traceability and fast restoration. Review how deployment architecture supports rollback through model registries and artifact versioning, immutable storage for released models, blue/green deployment and progressive rollout, API routing controls and rollback automation in CI/CD pipelines, and monitoring systems integrated with rollback triggers.

2. Inspection of Evidence

   1. Assess documentation that describes the end-to-end rollback approach, including: component-specific procedures (e.g., model weights, safety filters, tokenizer files), rollback trigger definitions and thresholds, approval processes and safety team involvement, monitoring and validation steps before, during, and after rollback, and consumer communication and SLA considerations.

   2. Tools and Technical Infrastructure Assessment: Confirm the use and reliability of model registries and version control, artifact storage with integrity protection, API version management and routing, rollback integration in deployment pipelines, configuration rollback mechanisms for model parameters, and monitoring for rollback trigger conditions.

   3. Select a sample of recent or high-risk model versions and verify that predefined rollback plans exist, testing of rollback functionality has been performed, known good versions are identifiable and preserved, simulation exercises (e.g., red-teaming, failure drills) are conducted, and evaluation artifacts confirm rollback effectiveness.

   4. Review of Executed Rollbacks: Where rollbacks have occurred, confirm documentation of triggers and root causes; validate alignment with defined rollback criteria; verify execution steps, timeline, and authority approval; assess post-rollback evaluations and consumer impact reports; and review resulting improvements or preventive measures.

   5. Monitoring and Automated Rollback Integration: Evaluate how the organization uses automated monitoring for performance drops or safety breaches, user feedback and telemetry to detect potential rollback triggers, integration with CI/CD for rollback automation, and progressive deployment and automated reversion capabilities.

   6. Assess the clarity and effectiveness of communication practices during rollbacks, including notification protocols for affected users, status updates and expected impact communication, post-rollback confirmations and guidance, and root cause disclosure and mitigation summaries.

3. Evaluate the overall effectiveness of the rollback strategy, confirming that rollbacks restore models to safe, performant known good states; recovery times meet defined objectives for critical model issues; disruptions to consumers are minimized and well-communicated; known good states are reliably preserved, tested, and accessible; and documentation is clear, current, and technically accurate.

   1. Evaluate whether continuous improvement mechanisms are in place, including: regular review of rollback metrics, lessons learned from incidents, feedback loops from consumers, technical enhancements to rollback tooling, and evolving safety and performance evaluation criteria.

CEK: Cryptography, Encryption & Key Management

CEK-01: Encryption and Key Management Policy and Procedures

Control Specification

Establish, document, approve, communicate, apply, evaluate and maintain policies and procedures for Cryptography, Encryption and Key Management. Review and update the policies and procedures at least annually, or upon significant changes.

Auditing Guidelines for Model Providers (MP)

1. Policy Examination: Verify that the MP’s Cryptography, Encryption, and Key Management (CEK) policy exists and addresses the planning, delivery, and support of cryptographic functions throughout the model lifecycle. Confirm that the policy includes protection of model weights, training data, and fine-tuning prompts; specifications for key generation, rotation, and destruction; requirements for encryption algorithm standards and storage mechanisms; and enforcement of role-based access controls for CEK operations within the model environment.

2. Governance: Confirm that the CEK policy is formally approved by senior management and that approval is documented through official means such as a policy register, governance meeting minutes, or executive signoff. Verify that ownership of the policy is assigned to a designated role or function and that the policy is reviewed and updated at least annually or in response to significant changes. Such changes may include the deployment of new models or model versions, migration to updated machine learning infrastructure, introduction of new cryptographic services, or updates to regulatory and privacy compliance expectations.

3. Communication: Review records showing that the CEK policy has been communicated to relevant stakeholders including ML engineers, platform security teams, data governance specialists, and operational staff. Acceptable forms of evidence may include internal communications, attestation logs, or training attendance records.

4. Implementation Validation: Validate the implementation of the CEK policy by inspecting encryption configurations protecting model artifacts, usage logs of keys applied during model inference or deployment, encryption controls over training and inference data pipelines, and secure storage practices for fine-tuning and prompt data.

5. Role Assignment: Review the policy and supporting documentation to confirm that CEK responsibilities are clearly assigned for tasks such as model encryption, secure deployment configuration, key lifecycle management, and access review exception handling. Ensure that these responsibilities are tied to named individuals or well-defined functional teams within the MP’s organization.

6. Training and Awareness: Inspect training plans, onboarding procedures, or continuing education programs to confirm that personnel responsible for cryptographic operations related to the model lifecycle have been trained on the CEK policy and its implementation.

7. Compliance Monitoring: Evaluate whether the MP monitors compliance with its CEK policy through controls such as detection of unencrypted model interactions, audits of training data encryption, logging of model artifact access, and periodic review of runtime encryption configurations.

8. Upstream and Downstream Dependencies: Review how the MP accounts for upstream expectations from data providers and downstream responsibilities toward OSPs or APs. Confirm that the CEK policy either includes explicit guarantees (e.g., contractual encryption clauses) or identifies unmitigated risks and outlines monitoring and mitigation practices to ensure secure handling of model-related data and artifacts.

CEK-02: CEK Roles and Responsibilities

Control Specification

Define and implement cryptographic, encryption and key management roles and responsibilities.

Auditing Guidelines for Model Providers (MP)

1. Verify that MP roles and responsibilities are defined in formal policies and procedures for cryptographic, encryption, and key management operations (e.g., model protection, key lifecycle management, access governance).

2. Confirm that AI-specific responsibilities are defined in alignment with the MP’s role (e.g., securing model weights, training data, inference results, and encrypted model-serving endpoints) and that role assignments are documented and maintained.

3. Review documentation to confirm that responsibilities are mapped to roles across platform, infrastructure, and ML engineering teams.

4. Validate that model-specific asks (e.g., securing checkpoints, protecting prompts and outputs) are assigned to appropriate functional owners.

5. Verify that segregation of duties is enforced between model developers, infrastructure admins, and personnel managing CEK systems.

6. Confirm that all personnel with responsibilities receive training relevant to cryptographic controls across the model lifecycle.

7. Verify that role assignments are reviewed at least annually or after major changes to model architecture, hosting infrastructure, or compliance obligations.

8. Confirm that governance structures oversee roles and periodically assess role clarity, risk alignment, and operational independence.

9. Validate that continuity planning exists for roles supporting model operations, including backups for encryption key handlers and model security leads.

10. Verify that responsibilities include coordination with upstream data providers and downstream consumers (e.g., OSPs, APs) to ensure encryption expectations are clearly assigned and met.

CEK-03: Data Protection

Control Specification

Provide data protection at-rest, in-transit and, where applicable, in-use by using cryptographic libraries certified to approved standards.

Auditing Guidelines for Model Providers (MP)

1. Verify that MP encryption is implemented for sensitive data at rest, including model weights, training datasets, logs, and artifacts, using approved algorithms (e.g., AES-256).

2. Verify that encryption is enforced for data in transit across internal model workflows and external access points using secure transport protocols (e.g., TLS 1.2+).

3. Confirm that sensitive data types requiring encryption are clearly identified across the model lifecycle (e.g., training, fine-tuning, inference).

4. Review the data classification framework and ensure encryption requirements are appropriately mapped to AI-related data types based on sensitivity.

5. Validate that cryptographic libraries used in model training, deployment, and runtime environments are certified to recognized standards (e.g., FIPS 140-2/3).

6. Verify that encryption keys protecting model-related data are managed through a centralized key management system with access control, logging, and lifecycle enforcement.

7. Confirm that encryption settings across model development and inference environments are hardened (e.g., strong cipher suites, disabled legacy protocols).

8. Verify that any exceptions to encryption requirements (e.g., in third-party tools, in pipelines) are documented, risk-assessed, and approved with compensating controls in place.

9. Confirm that monitoring and alerting mechanisms are in place to detect unencrypted model data transfers, failed encryption events, or misconfigurations.

10. Verify that encryption is applied to inference inputs and outputs exchanged with external systems and that model-serving endpoints enforce transport encryption.

11. Validate that encryption keys used for securing inference APIs, runtime artifacts, and logging pipelines are governed under the same key management policies as the rest of the model platform.

12. Review upstream data source agreements and downstream consumer integrations to ensure encryption requirements are consistently applied and verifiable end-to-end.

CEK-04: Encryption Algorithm

Control Specification

Utilize encryption algorithms following industry standards for protecting data, based on the data classification and associated risks.

Auditing Guidelines for Model Providers (MP)

1. Verify that the MP maintains documented standards for approved encryption algorithms mapped to the sensitivity of model-related assets (e.g., training data, model weights, inference outputs).

2. Confirm that selected algorithms meet current cryptographic best practices and are used to protect critical components, including encrypted storage of models, secure inference pipelines, and logging of AI interactions.

3. Review whether encryption algorithms are periodically assessed for strength, deprecation, or vulnerabilities and that obsolete algorithms are phased out in a timely and controlled manner.

4. Validate that algorithm selection accounts for usability within model-serving infrastructure, including performance constraints in high-throughput inference workloads or distributed AI processing.

5. Confirm that encryption is consistently applied across the model lifecycle, from training data ingestion and checkpoint storage to deployment, inference, and API exposure.

6. Verify that cryptographic operations using these algorithms are integrated with robust key management systems that enforce proper scoping, access control, and tenant isolation.

7. Review whether encryption standards are approved or overseen by a governance function such as an ML security committee, cryptography board, or internal control authority.

8. Confirm that any third-party components used in the model stack (e.g., pre-processing tools, AI SDKs, model routers) use algorithms that align with MP-approved cryptographic standards.

9. Validate that vulnerability management processes (e.g., CVE tracking, code audits, third-party library reviews) identify and respond to risks related to algorithm misuse or weakness.

10. Verify that the MP considers both upstream requirements (e.g., encryption of source datasets or training inputs) and downstream integrations (e.g., AP/OSP consumers) when selecting or updating encryption algorithms.

CEK-05: Encryption Change Management

Control Specification

Establish a standard change management procedure, to accommodate changes from internal and external sources, for review, approval, implementation and communication of cryptographic, encryption and key management technology changes.

Auditing Guidelines for Model Providers (MP)

1. Verify that the MP maintains a documented change management procedure governing CEK-related changes across model training infrastructure, encrypted storage, and inference delivery platforms.

2. Confirm that the procedure accommodates both internally driven changes (e.g., model security upgrades, new key isolation mechanisms) and externally required changes (e.g., deprecation of crypto libraries, compliance mandates).

3. Review whether CEK changes are formally requested, logged, and reviewed through an established platform or security change governance process.

4. Verify that roles and responsibilities are defined for change review, approval, implementation, and monitoring, including platform engineering, ML security, compliance, and data governance.

5. Confirm that each approved CEK change includes a formal implementation plan with testing requirements, rollback mechanisms, and change communication milestones.

6. Review how CEK changes are communicated to internal ML teams and to external parties (e.g., APs, OSPs) who depend on consistent model cryptographic behavior.

7. Validate that version control is applied to model encryption configurations, protected storage formats, and inference endpoint security profiles.

8. Verify that CEK changes are validated post-implementation using metrics such as model availability, encryption status of inference data, or key rotation logs.

9. Confirm that change records are retained, including evidence of review, testing, approvals, and fallback planning, and confirm that they are available for security and audit reviews.

10. Review whether CEK changes account for upstream cryptographic components (e.g., cloud KMS, model pre-processing) and downstream service consumers (e.g., APs, OSPs) to ensure compatibility and reliability across the full delivery pipeline.

CEK-06: Encryption Change Cost Benefit Analysis

Control Specification

Manage and adopt changes to cryptography-, encryption-, and key management-related systems (including policies and procedures) that fully account for downstream effects of proposed changes, including residual risk, cost, and benefits analysis.

Auditing Guidelines for Model Providers (MP)

1. Verify that the MP maintains a documented process for managing changes to CEK systems affecting model training pipelines, deployment infrastructure, and runtime inference layers.

2. Confirm that CEK changes (e.g., algorithm upgrades, key lifecycle automation, model checkpoint encryption) are reviewed by cryptographic or architecture governance teams before implementation.

3. Review whether each CEK change is supported by a cost-benefit analysis addressing potential gains (e.g., confidentiality, performance, regulatory alignment) and costs (e.g., compute impact, storage, vendor lock-in).

4. Validate that residual risks are documented and evaluated as part of change justification, particularly when weaker cryptographic mechanisms must be temporarily tolerated.

5. Confirm that CEK changes are evaluated for downstream impact on APs or OSPs that consume the model (e.g., interface compatibility, key management expectations, latency implications).

6. Verify that internal stakeholders, including ML engineers, platform teams, and compliance, are consulted and approve CEK changes, especially for production-grade model deployments.

7. Review whether version tracking, rollback procedures, and architectural documentation are maintained for cryptographic updates to model storage, inference APIs, or supporting services.

8. Validate that CEK-related changes are monitored post-deployment for signs of regression, including inference degradation, model access issues, or key rotation failures.

9. Confirm that learnings from prior CEK changes are collected and used to refine future risk/benefit analysis templates, especially where past changes caused operational friction.

10. Verify that upstream dependencies (e.g., cryptographic libraries, key management systems) and downstream integrations (e.g., AP access layers, OSP access layers) are considered and validated before cryptographic changes are finalized.

CEK-07: Encryption Risk Management

Control Specification

Establish and maintain an encryption and key management risk program that includes provisions for risk assessment, risk treatment, risk context, monitoring, and feedback.

Auditing Guidelines for Model Providers (MP)

1. Verify that the MP has a documented CEK risk management program that addresses cryptographic controls across model development, deployment, and inference environments.

2. Confirm that CEK risks are contextualized based on model training data sensitivity, model weights, version control, inference results, and access to model endpoints.

3. Review the methodology for CEK risk assessment, including how encryption coverage, key lifecycle integrity, and AI data classification are considered in risk scoring.

4. Verify that a CEK-specific risk register or comparable tracking system includes identified threats, vulnerabilities, and mitigation plans tied to model hosting or training infrastructure.

5. Confirm that treatment strategies address risks such as unsecured model checkpoints, weak key isolation for tenant data, or insufficient controls over prompt history retention.

6. Validate that residual CEK risks are periodically reviewed by an appropriate governance structure (e.g., ML security board, cryptography review group) and re-evaluated after model or infrastructure changes.

7. Review how CEK risks are monitored through mechanisms such as encrypted storage enforcement, key use analytics, and model-serving audit trails.

8. Confirm that incident response, audit results, and internal reviews feed into the CEK risk program and lead to updates in encryption policy or model protection architecture.

9. Verify that CEK risks unique to AI operations, such as exposure of sensitive embeddings, inference leakage, or training data memorization, are included and assessed.

10. Validate that the MP evaluates upstream dependencies (e.g., training data providers, KMS services) and downstream consumers (e.g., APs or OSPs using the model), ensuring end-to-end CEK risk awareness and treatment.

CEK-08: Service Customer Key Management Capability

Control Specification

Service providers must provide the capability for service customers to manage their own data encryption keys.

Auditing Guidelines for Model Providers (MP)

1. Verify that the MP provides secure APIs or management interfaces that allow the AIC to manage their encryption keys when using hosted or fine-tuned models, if key management is offered as part of the service.

2. Confirm that the MP ensures AIC-supplied or tenant-scoped keys are exclusively used to protect that AIC’s data, including prompts, inference outputs, logging data, and model tuning artifacts.

3. Validate that the MP provides key usage logging for AIC-managed keys (e.g., model access, encryption, signing), and makes this information accessible to the AIC through dashboards, APIs, or audit reports.

4. Confirm that the MP provides contracts or platform service terms that explicitly define the AIC’s ability to supply, rotate, revoke, or monitor encryption keys related to the use of hosted or fine-tuned models.

5. Verify that the MP assigns responsibilities for enabling and maintaining AIC-managed key support to appropriate teams (e.g., platform engineering, cryptography services, ML infrastructure).

6. Confirm that the MP defines exception handling processes for model services or tooling that cannot yet support AIC-managed keys, including notification, mitigation, and documentation of limitations.

7. Validate that the MP allows key control capabilities to be tested by the AIC before deploying sensitive models, including secure validation of prompt encryption, runtime isolation, and log protections.

8. Verify that the MP subjects AIC-managed key control support to periodic review by CEK governance or engineering leadership to reflect changes in cryptographic best practices or platform capabilities.

9. Confirm that the MP enforces downstream AIC key control requirements, including tenant-level encryption, key isolation, and clear accountability for cryptographic operations involving AIC data.

10. Validate that the MP periodically assesses upstream dependencies (e.g., KMS integrations, shared model pipelines, data encryption modules) to ensure continued support for AIC-managed key configurations and controls.

CEK-09: Encryption and Key Management Audit

Control Specification

Audit encryption and key management systems, policies, and processes with a frequency that is proportional to the risk exposure of the system with audit occurring preferably continuously but at least annually and after any security event(s).

Auditing Guidelines for Model Providers (MP)

1. Verify that the MP encryption and key management systems, policies, and processes are audited at a frequency that reflects the associated risk exposure preferably continuously but at least annually and after any security event.

2. Confirm that audits are triggered after changes to model development pipelines, training infrastructure, or cryptographic service integration.

3. Review the audit scope to ensure it covers CEK across model storage, training environments, runtime encryption, and data protection layers.

4. Validate that audit procedures evaluate compliance with internal policies and external frameworks (e.g., NIST, ISO), focusing on algorithm strength, secure RNGs, and key protection.

5. Verify that CEK audits are performed independently from development, training, or infrastructure teams managing the model lifecycle.

6. Confirm that audit results are recorded, reviewed, and acted upon, with timelines and remediation tracking established for any CEK-related findings.

7. Review communication of audit results and residual risks to internal teams (e.g., security engineering, MLOps, legal, compliance).

8. Verify that automation and monitoring tools (e.g., key usage logs, training pipeline validators) support continuous auditability of CEK controls.

9. Confirm that encryption and key protections for AI model inputs, outputs, and checkpoints are within scope of CEK audit activities.

10. Validate that CEK audit procedures are reviewed and updated based on changes to cryptographic standards, AI platform risk posture, and interdependencies with downstream consumers (e.g., OSPs, APs) who rely on model-related key protections.

CEK-10: Key Generation

Control Specification

Generate Cryptographic keys using industry accepted cryptographic libraries specifying the algorithm strength and the random number generator used.

Auditing Guidelines for Model Providers (MP)

1. Verify that the MP uses approved, standards-based cryptographic libraries (e.g., FIPS 140-2/3 certified) to generate encryption keys that protect model-related assets, such as training data, model weights, inference outputs, and deployment artifacts.

2. Confirm that key generation processes explicitly specify algorithm strength and type based on data sensitivity (e.g., training data vs. inference outputs).

3. Validate that cryptographic random number generators (RNGs) used for key generation meet approved standards (e.g., NIST SP 800-90A).

4. Verify that key generation is securely integrated into the model development, deployment, and inference environments (e.g., secured build systems, MLOps pipelines).

5. Review access control enforcement to ensure only authorized ML engineers or automation systems can initiate key generation.

6. Confirm that AI model–specific data (e.g., model weights, checkpoints, fine-tuning data) is protected using keys generated through approved methods.

7. Verify that keys are not embedded or hardcoded into code repositories, model containers, or inference runtimes.

8. Review audit logs capturing key generation activities, including user, system, purpose, and timestamp.

9. Confirm that test and research environments use keys and RNGs that are logically and cryptographically separate from production models.

10. Validate that key generation processes are periodically reviewed and updated to reflect cryptographic best practices, AI architecture changes, and coordination requirements with downstream consumers (e.g., OSPs, APs) relying on model-related key protections.

CEK-11: Key Purpose

Control Specification

Manage cryptographic secret and private keys that are provisioned for a unique purpose.

Auditing Guidelines for Model Providers (MP)

1. Verify that the MP assigns cryptographic keys and secrets (e.g., model signing keys, API tokens, training data access credentials) to a unique purpose and that purpose separation is maintained across model training, storage, inference, and key management systems.

2. Confirm that documentation exists mapping each key to its specific function within the model pipeline and that this mapping is reviewed periodically for completeness and accuracy.

3. Verify that technical and procedural controls prevent the same key from being used for multiple cryptographic purposes (e.g., using one key for both model signing and inference output encryption).

4. Review model infrastructure and storage configurations to ensure that keys are used only within their intended scope and are not shared across unrelated processes.

5. Confirm that access to each key is restricted based on assigned responsibilities (e.g., ML engineers, deployment teams) and reflects least privilege principles.

6. Validate that keys used to protect AI-specific artifacts (e.g., model weights, tuning datasets, output tokens) are purpose-bound and not repurposed across unrelated AI components.

7. Review whether key metadata includes attributes or tags specifying key purpose and that purpose enforcement is applied across MLOps or cryptographic tooling.

8. Confirm that CI/CD or model deployment pipelines enforce key segregation and reject configurations where keys are assigned conflicting or undefined purposes.

9. Verify that logs are collected for key usage events and that anomalous or unauthorized use of keys outside their assigned purpose is detectable and reviewable.

10. Confirm that keys shared with downstream consumers (e.g., OSPs, APs) are scoped to specific tasks (e.g., payload validation) and are not used across multiple interaction points or services.

CEK-12: Key Rotation

Control Specification

Rotate cryptographic keys in accordance with the calculated cryptoperiod, which includes provisions for considering the risk of information disclosure and legal and regulatory requirements.

Auditing Guidelines for Model Providers (MP)

1. Verify that the MP rotates cryptographic keys in accordance with defined cryptoperiods, considering the risk of information disclosure and legal or regulatory requirements that apply to model storage, inference operations, and ML infrastructure.

2. Confirm that key rotation procedures are documented and updated to reflect the latest cryptographic standards, risk assessments, and applicable legal or regulatory changes.

3. Verify that key rotation is implemented using automated scheduling or secure tooling within the MLOps pipeline or infrastructure-as-code deployments.

4. Review system configurations to ensure that key rotation is enforced for environments managing sensitive model operations, including checkpoints, weights, and prompt handling.

5. Confirm that access to configure or execute key rotation is restricted to authorized ML platform or security personnel, and that such actions require approval.

6. Validate that keys tied to AI operations (e.g., inference payload encryption, secure model deployment) are rotated regularly and separately from infrastructure keys.

7. Review logs and audit records of key rotation events to ensure they capture identity, time, systems involved, and compliance with cryptoperiod policies.

8. Confirm that newly rotated keys are propagated securely across model-serving infrastructure and supporting services without service interruption or exposure.

9. Verify that decommissioned or superseded keys are archived or destroyed securely based on data classification, contractual terms, and retention policies.

10. Confirm that key rotation schedules or dependencies are coordinated with downstream consumers (e.g., OSPs or APs) to ensure operational continuity and secure data exchange.

CEK-13: Key Revocation

Control Specification

Define, implement and evaluate processes, procedures and technical measures to revoke and remove cryptographic keys prior to the end of its established cryptoperiod, when a key is compromised, or an entity is no longer part of the organization, which include provisions for legal and regulatory requirements.

Auditing Guidelines for Model Providers (MP)

1. Verify that the MP defines, implements, and evaluates processes, procedures, and technical measures to revoke cryptographic keys prior to the end of their cryptoperiod, including in cases of model deprecation, infrastructure changes, or key compromise, in accordance with legal and regulatory requirements.

2. Confirm that key revocation policies and procedures exist and are reviewed regularly to reflect model lifecycle changes, evolving threats, and legal or regulatory requirements.

3. Verify that key revocation processes are enforced using secure, automated methods in MLOps pipelines, model registries, and cryptographic frameworks where feasible.

4. Review deployment, training, and serving infrastructure to ensure revoked keys are immediately and securely removed from model containers, checkpoints, and config files.

5. Confirm that revocation rights are restricted to authorized roles (e.g., platform security, cryptographic governance), and that all revocation activities are subject to logging and/or approval mechanisms.

6. Validate that keys used for model-specific purposes (e.g., encrypted weights, signing model versions, securing inference outputs) are revoked in accordance with security or operational triggers and removed from all stages of the model pipeline.

7. Review key revocation logs to verify that revocation activities are timestamped, traceable, and correlated to triggering events.

8. Confirm that model-serving environments and inference APIs properly handle key revocation by switching to replacement keys or rejecting unauthorized access.

9. Verify that revoked keys are securely archived or destroyed following organizational policies and that this process considers contractual and jurisdictional retention rules.

10. Confirm that key revocation impacting downstream parties (e.g., OSPs or APs) is coordinated and communicated clearly, ensuring secure handoff or service continuity.

CEK-14: Key Destruction

Control Specification

Define, implement, and evaluate processes, procedures, and technical measures to securely destroy cryptographic keys when they are no longer needed, which include provisions for legal and regulatory requirements.

Auditing Guidelines for Model Providers (MP)

1. Verify that the MP defines, implements, and evaluates processes, procedures, and technical measures to destroy cryptographic keys stored outside a secure environment and to revoke keys stored in Hardware Security Modules (HSMs) when they are no longer needed, including those associated with model training datasets, inference engines, and version-controlled model artifacts.

2. Verify that the MP has a documented policy and process for the secure deletion of cryptographic keys and associated data once they are no longer needed, ensuring compliance with relevant legal, regulatory, and contractual requirements for data and key destruction.

3. Confirm that the criteria for key destruction and revocation include model retirement, infrastructure decommissioning, cryptoperiod expiry, or regulatory obligations.

4. Verify that destruction methods for software-stored keys follow industry standards (e.g., cryptographic erasure, zeroization) and are executed through secure tooling or automated processes.

5. Review model development and deployment environments (e.g., MLOps pipelines, training clusters) to confirm that temporary or residual keys are removed from memory, containers, and configuration files.

6. Confirm that HSM- or KMS-managed keys are securely revoked and made unrecoverable once models or their associated data flows no longer require protection.

7. Validate that model-specific keys (e.g., encrypting weights, securing checkpoints, output signing) are revoked or destroyed when the models they protect are deprecated or replaced.

8. Review audit logs or revocation records to verify key destruction activities, ensuring they capture key ID, source system, revocation method, and responsible user.

9. Confirm that destruction activities are integrated into the MP’s model lifecycle workflows, including automated cleanup during CI/CD operations or environment resets.

10. Verify that key destruction practices adhere to legal and regulatory mandates concerning cryptographic asset disposal, intellectual property protection, and jurisdictional data handling.

11. Confirm that revocation or destruction of shared model-related keys (e.g., used by OSPs or APs) is communicated and coordinated to preserve service continuity and cryptographic integrity.

CEK-15: Key Activation

Control Specification

Define, implement and evaluate processes, procedures and technical measures to create keys in a pre-activated state when they have been generated but not authorized for use, which include provisions for legal and regulatory requirements.

Auditing Guidelines for Model Providers (MP)

1. Verify that the MP defines processes, procedures, and technical measures to generate cryptographic keys in a pre-activated state, where keys are not authorized for use until explicitly approved.

2. Confirm that pre-activated keys are securely stored and logically separated from active key inventories until explicitly activated (e.g., within model training environments, inference pipelines, or MLOps automation).

3. Review the MP’s approval workflow to validate that a formal authorization step is required before key activation, including references to system roles or automated triggers.

4. Validate that access to key activation functions is restricted to authorized personnel or systems and that multi-party approval is enforced where required.

5. Confirm that activation policies are consistently applied across all AI environments, including dev/test, staging, production, and model registry operations.

6. Review activation event logs and ensure they capture key identifiers, purpose, activation status, initiating user or system, and timestamp.

7. Verify that legal and regulatory requirements related to model security, data privacy, or sector-specific encryption rules (e.g., HIPAA, GDPR) are integrated into key activation procedures.

8. Confirm that pre-activated keys include policies for timeout, expiration, or lifecycle management in the absence of activation.

9. Verify that model-serving keys (e.g., used for secure inference requests or encryption of outputs) are also subject to pre-activation policies and tied to role-specific approval.

10. Review whether MP activation procedures address upstream key dependencies (e.g., data ingestion sources) and downstream handoff requirements to OSPs and APs for coordinated activation readiness.

CEK-16: Key Suspension

Control Specification

Define, implement and evaluate processes, procedures and technical measures to monitor, review and approve key transitions from any state to/from suspension, which include provisions for legal and regulatory requirements.

Auditing Guidelines for Model Providers (MP)

1. Verify that the MP defines processes, procedures, and technical measures to monitor, review, and approve cryptographic key transitions to and from suspension, including keys associated with model training, inference, and secure model distribution.

2. Confirm that the MP defines valid criteria for suspending cryptographic keys, including policy violations, security anomalies, temporary operational pauses, and other risk-based triggers relevant to model lifecycle protection and inference security.

3. Review documentation that defines how suspension of cryptographic keys is authorized and tracked through change management or security governance workflows.

4. Validate that suspended keys are isolated from operational AI pipelines while remaining recoverable for future reactivation.

5. Verify that key suspension and reactivation capabilities are restricted to designated personnel or systems, and that separation of duties is enforced between development, security, and infrastructure roles.

6. Review monitoring tools and anomaly detection systems that alert on suspicious activity that may result in or follow from key suspension events.

7. Verify that audit logs capture cryptographic key suspension details, including initiating user/system, time, model asset affected, and justification for the suspension.

8. Confirm that legal, privacy, and industry-specific regulatory considerations (e.g., export controls, healthcare model protections) are integrated into the MP’s key suspension procedures.

9. Validate that cryptographic keys securing sensitive model assets (e.g., model weights, training data, fine-tuning prompts) are covered by suspension governance policies.

10. Review whether MP key suspension procedures address upstream key provisioning from data sources and downstream coordination with OSPs and APs depending on model cryptographic assurances.

CEK-17: Key Deactivation

Control Specification

Define, implement and evaluate processes, procedures and technical measures to deactivate keys at the time of their expiration date, which include provisions for legal and regulatory requirements.

Auditing Guidelines for Model Providers (MP)

1. Verify that the MP defines, implements, and evaluates processes, procedures, and technical measures to deactivate cryptographic keys at the time of their expiration date, including provisions for legal and regulatory requirements (e.g., keys used for securing model weights, training pipelines, or inference output).

2. Confirm that all cryptographic keys protecting model training data, artifacts, or inference systems are assigned expiration dates and recorded in the MP’s key tracking systems.

3. Review whether key expiration triggers are integrated with deployment and runtime environments to enforce automated deactivation.

4. Validate that expired keys are excluded from model serving systems, CI/CD pipelines, and secure storage endpoints.

5. Confirm that access to deactivated keys is revoked across all MP environments, including training clusters and inference runtimes.

6. Review audit records capturing key expiration and deactivation events, ensuring completeness of metadata (timestamp, key ID, purpose, and affected model).

7. Verify that expired keys are either archived securely or marked for revocation and destruction per MP retention policies.

8. Confirm that the MP’s key deactivation procedures address legal and regulatory requirements, including those related to data confidentiality, model governance, and IP protection.

9. Validate that cryptographic keys used in model-specific AI operations (e.g., encrypting training prompts, inference results) are covered by the deactivation lifecycle.

10. Review whether the MP coordinates key deactivation timing and transitions with upstream data providers and downstream OSPs or APs that depend on model cryptographic integrity.

CEK-18: Key Archival

Control Specification

Define, implement and evaluate processes, procedures and technical measures to manage archived keys in a secure repository requiring least privilege access, which include provisions for legal and regulatory requirements.

Auditing Guidelines for Model Providers (MP)

1. Verify that the MP defines, implements, and evaluates processes, procedures, and technical measures to manage archived cryptographic keys in a secure repository requiring least privilege access, including provisions for legal and regulatory requirements (e.g., keys associated with model weights, training prompts, or inference checkpoints).

2. Confirm that archived keys are encrypted, segregated from active use, and access is limited to approved roles responsible for model lifecycle management or cryptographic assurance.

3. Review whether access to archived keys is governed by least privilege principles and limited to designated roles responsible for model lifecycle governance or compliance.

4. Validate that access to archived keys requires formal approval and is logged with details including requester, timestamp, and access purpose.

5. Confirm that archived keys cannot be reused for active encryption or model-serving operations and are logically separated from active keys.

6. Review whether archived keys are retained based on a defined schedule that considers regulatory requirements, customer agreements, and IP protection policies.

7. Verify that the MP regularly evaluates the need to retain archived keys and performs audits to identify expired or unnecessary keys.

8. Confirm that technical safeguards are implemented to prevent unauthorized export, duplication, or restoration of archived keys.

9. Validate that keys used in protecting model data (e.g., weights, training prompts, inference logs) are included in archival processes when retired or rotated.

10. Review whether the MP aligns archival practices with upstream data providers and downstream orchestration and application services to ensure retention compatibility and risk transparency.

CEK-19: Key Compromise

Control Specification

Define, implement and evaluate processes, procedures and technical measures to use compromised keys to encrypt information only in controlled circumstance, and thereafter exclusively for decrypting data and never for encrypting data, which include provisions for legal and regulatory requirements.

Auditing Guidelines for Model Providers (MP)

1. Verify that the MP defines, implements, and evaluates processes, procedures, and technical measures for handling compromised cryptographic keys, including provisions for legal and regulatory requirements.

2. Verify that the MP’s incident response plan includes documented steps for handling compromised keys, ensuring that the keys are restricted to decryption, securely revoked, and that they cannot be reused for encryption. Confirm that the MP’s incident response procedures comply with relevant legal and regulatory requirements for key management in the event of a compromise.

3. Confirm that the MP restricts use of compromised keys to decrypt-only operations under controlled circumstances (e.g., model checkpoint recovery, decryption of legacy training artifacts), and explicitly prohibits further encryption with such keys unless formally approved.

4. Review the MP’s method of identifying and flagging compromised keys within their cryptographic systems or model-serving infrastructure.

5. Validate that compromised keys are isolated from model training, inference, and deployment pipelines, and retained only for legacy decryption of prior data (e.g., model checkpoints, training inputs).

6. Confirm that access to compromised keys is restricted to authorized personnel under enhanced controls, such as break-glass procedures or time-bound access.

7. Review logs and event records capturing when, where, and by whom compromised keys were accessed, including AI workload context (e.g., prompt processing, model weight retrieval).

8. Verify that decrypt-only actions using compromised keys are tracked and documented with justification, referencing model lifecycle events when applicable.

9. Confirm that MP practices account for regulatory obligations related to cryptographic key handling in AI model development, including data privacy, IP protection, or industry-specific requirements.

10. Validate that compromised keys are not used in inference environments that process sensitive user data or AI-generated outputs without re-keying safeguards.

11. Review whether the MP coordinates with upstream data providers or downstream OSPs and APs to manage risks associated with compromised model-related keys.

CEK-20: Key Recovery

Control Specification

Define, implement and evaluate processes, procedures and technical measures to assess the risk to operational continuity versus the risk of the keying material and the information it protects being exposed if control of the keying material is lost, which include provisions for legal and regulatory requirements.

Auditing Guidelines for Model Providers (MP)

1. Verify that the MP defines, implements, and evaluates processes, procedures, and technical measures to assess the tradeoff between operational continuity and the risk of key exposure in the event of keying material loss, including provisions for legal and regulatory requirements.

2. Confirm that the MP conducts periodic risk assessments that evaluate key recovery scenarios across model development and inference environments (e.g., encrypted weights, training data keys, model signing credentials) and considers the risk to model confidentiality, availability, and downstream dependencies.

3. Review whether the MP classifies keys based on their role in model development, deployment, and protection, and includes each class in recovery planning (e.g., model encryption keys, developer API keys, MLOps signing keys).

4. Validate that key recovery mechanisms are in place for critical model infrastructure and that key backups are securely stored, encrypted, and protected against unauthorized access or tampering.

5. Confirm that recovery procedures are routinely tested and include provisions for restoring access to AI model environments without introducing vulnerabilities or exposing confidential training materials.

6. Review MP policies for authorizing key recovery events, including documentation of business justification, escalation protocols, and secure restoration practices.

7. Verify that access to key recovery systems is governed by strict access controls, separation of duties, and continuous logging to detect any unauthorized usage.

8. Confirm that the MP evaluates the business and security impacts of unrecoverable key loss (e.g., model loss or retraining requirements) as part of its risk management process.

9. Validate that model-specific encryption keys protecting training data, inference logs, or embeddings are covered in the recovery plan, and that re-keying strategies are considered post-recovery to preserve integrity.

10. Review whether the MP communicates key recovery constraints and dependencies to downstream OSPs and APs, ensuring that inherited keys and trust relationships are clearly documented and supported.

CEK-21: Key Inventory Management

Control Specification

Define, implement and evaluate processes, procedures and technical measures in order for the key management system to track and report all cryptographic materials and changes in status, which include provisions for legal and regulatory requirements.

Auditing Guidelines for Model Providers (MP)

1. Verify that the MP defines, implements, and evaluates processes, procedures, and technical measures to ensure the key management system can track and report all cryptographic materials and changes in key status.

2. Confirm that the MP’s key management system maintains a complete and up-to-date inventory of all cryptographic keys and materials in scope, including key attributes (e.g., type, status, owner, lifecycle stage, algorithm) and usage context.

3. Review whether the inventory includes metadata for each key such as type, usage purpose, algorithm, owner, associated models or datasets, cryptoperiod, and current status.

4. Validate that the system automatically logs changes to key status (e.g., activation, suspension, expiration, destruction, compromise) with timestamps and source identifiers.

5. Confirm that access to key inventory records is restricted to authorized roles, and that controls are in place to prevent unauthorized modification.

6. Review whether MP policies require long-term retention of inventory records related to regulated data or proprietary model artifacts.

7. Verify that model-specific encryption keys (e.g., for model weights, prompts, logs, and checkpoints) are tracked separately from general infrastructure keys.

8. Confirm that key inventory events are monitored and correlated with other security logs to detect unauthorized access or anomalous activity.

9. Validate that inventory reviews are conducted periodically to ensure completeness, accuracy, and alignment with current model deployments and services.

10. Review whether the MP shares key inventory status updates with downstream OSPs or APs when relevant to shared infrastructure or cryptographic dependency management.

DCS: Datacenter Security

DCS-01: Physical and Environmental Security Policy and Procedures

Control Specification

Establish, document, approve, communicate, apply, evaluate and maintain policies and procedures for physical and environmental security. Review and update the policies and procedures at least annually, or upon significant changes.

Auditing Guidelines for Model Providers (MP)

1. Verify that physical and environmental security policies apply to all facilities supporting model training, inference operations, and storage of model artifacts.

2. Verify that evidence demonstrates controlled facility access, appropriate environmental protection measures, and periodic (at least annually or upon significant changes) review of the policies.

3. Verify that when third‑party facilities are used, assurance documentation exists to validate the effectiveness of their physical security controls.

DCS-02: Off-Site Equipment Disposal Policy and Procedures

Control Specification

Establish, document, approve, communicate, apply, evaluate and maintain policies and procedures for the secure disposal of equipment used outside the organization’s premises. If the equipment is not physically destroyed a data destruction procedure that renders recovery of information impossible must be applied. Review and update the policies and procedures at least annually, or upon significant changes.

Auditing Guidelines for Model Providers (MP)

1. Verify the MP’s policy requires that any physical media used for transferring or storing training data is securely disposed of or sanitized after use.

2. Review contracts with CSPs to ensure they are obligated to follow secure disposal standards for infrastructure used in model training or hosting.

DCS-03: Off-Site Transfer Authorization Policy and Procedures

Control Specification

Establish, document, approve, communicate, apply, evaluate and maintain policies and procedures for the relocation or transfer of hardware, software, or data/information to an offsite or alternate location. The relocation or transfer request requires the written or cryptographically verifiable authorization. Review and update the policies and procedures at least annually, or upon significant changes.

Auditing Guidelines for Model Providers (MP)

1. Verify the MP has a documented policy requiring authorization for transferring model artifacts or large training datasets to off-site locations or different cloud providers.

2. Check for cryptographic verification (e.g., checksums, signatures) of assets upon transfer.

DCS-04: Secure Area Policy and Procedures

Control Specification

Establish, document, approve, communicate, apply, evaluate and maintain policies and procedures for maintaining a safe and secure working environment in offices, rooms, and facilities. Review and update the policies and procedures at least annually, or upon significant changes.

Auditing Guidelines for Model Providers (MP)

1. Verify the MP’s security policy covers the physical protection of any on-premises labs or data centers used for model research and development.

2. Confirm the MP reviews the physical security attestations (e.g., SOC 2) of the CSPs hosting their training and inference workloads.

DCS-05: Secure Media Transportation Policy and Procedures

Control Specification

Establish, document, approve, communicate, apply, evaluate and maintain policies and procedures for the secure transportation of physical media. Review and update the policies and procedures at least annually, or upon significant changes.

Auditing Guidelines for Model Providers (MP)

1. Verify the MP has a strict policy for the secure transport of physical media containing training datasets or model weights, requiring encryption and tamper-evident packaging.

2. Examine chain-of-custody logs for any such transfers.

DCS-06: Assets Classification

Control Specification

Classify and document the physical, and logical assets (e.g., applications) based on the organizational business risk. Review and update the assets’ classification at least annually, or upon significant changes.

Auditing Guidelines for Model Providers (MP)

1. Verify the MP classifies its assets, including training datasets, model weights, and research documentation, based on confidentiality and IP value.

2. Confirm that the classification scheme informs data handling, storage, and access control policies.

DCS-07: Assets Cataloguing and Tracking

Control Specification

Catalogue and track all relevant physical and logical assets located at all of the service provider’s sites within a secured system. Review and update the catalogue at least annually, or upon significant changes.

Auditing Guidelines for Model Providers (MP)

1. Verify the MP maintains a versioned inventory of all model artifacts, including training data, source code, and model weights, in a secure model registry.

2. Check that the inventory tracks the lineage and deployment status of each model.

DCS-08: Controlled Physical Access Points

Control Specification

Design and implement physical security perimeters to safeguard personnel, data, and information systems.

Auditing Guidelines for Model Providers (MP)

1. Verify that the MP has assessed the physical security of the data centers where model training and hosting occur by reviewing the CSP’s compliance certifications and reports.

2. Confirm that data residency for sensitive training data aligns with physically secure locations.

DCS-09: Equipment Identification

Control Specification

Use equipment identification as a method for connection authentication.

Auditing Guidelines for Model Providers (MP)

1. Verify that the MP uses secure methods to identify and authenticate infrastructure components (e.g., specific GPU nodes, storage servers) used in the training pipeline.

2. Confirm that only identified and authorized equipment can access sensitive training data.

DCS-10: Secure Area Authorization

Control Specification

Allow only authorized personnel access to secure areas, with all ingress and egress points restricted, documented, and monitored by physical access control mechanisms. Retain access control records on a periodic basis as deemed appropriate by the organization.

Auditing Guidelines for Model Providers (MP)

1. Verify the MP assesses the physical access controls of the data centers used for model training and research, particularly for facilities housing high-value compute resources.

2. Review the MP’s policies for securing any on-premises research labs.

DCS-11: Surveillance System

Control Specification

Implement, maintain, and operate datacenter surveillance systems at the external perimeter and at all the ingress and egress points to detect unauthorized ingress and egress attempts.

Auditing Guidelines for Model Providers (MP)

1. Verify the MP’s due diligence process confirms that CSPs hosting their sensitive model training and inference workloads have adequate surveillance systems.

2. Review the MP’s internal policies for surveillance of any on-premises research facilities.

DCS-12: Adverse Event Response Training

Control Specification

Train datacenter personnel to safely manage adverse events, including but not limited to unauthorized ingress and egress attempts.

Auditing Guidelines for Model Providers (MP)

1. Verify the MP’s due diligence process confirms that their CSPs train their data center personnel on responding to adverse physical events.

2. Review the MP’s own physical security policies for training requirements for staff at any on-premises research facilities.

DCS-13: Cabling Security

Control Specification

Define, implement and evaluate processes, procedures and technical measures that ensure a risk-based protection of power and telecommunication cables from a threat of interception, interference or damage at all facilities, offices and rooms.

Auditing Guidelines for Model Providers (MP)

1. Verify that the MP’s due diligence process confirms that their CSPs have robust cabling security to protect the integrity of the infrastructure used for model training and inference.

2. Review the MP’s internal policies for cabling security at any on-premises facilities.

DCS-14: Environmental Systems

Control Specification

Implement and maintain data center environmental control systems that monitor, maintain and test for continual effectiveness the temperature and humidity conditions within accepted industry standards.

Auditing Guidelines for Model Providers (MP)

1. Verify the MP’s due diligence confirms their CSPs have robust environmental controls, particularly for the high-density compute environments used for model training.

2. Assess the MP’s own business continuity plan for risks related to CSP environmental failures.

DCS-15: Secure Utilities

Control Specification

Secure, monitor, maintain, and test utilities services for continual effectiveness at planned intervals.

Auditing Guidelines for Model Providers (MP)

1. Verify the MP’s due diligence confirms their CSPs have redundant and secure utility services to support power-intensive model training workloads.

2. Assess the MP’s BC/DR plan for risks associated with utility failures at key training or inference locations.

DCS-16: Equipment Location

Control Specification

Keep business-critical equipment away from locations subject to high probability for environmental risk events.

Auditing Guidelines for Model Providers (MP)

1. Verify the MP’s policy for selecting cloud regions for model training and hosting considers environmental risks.

2. Confirm that critical model artifacts and training data are backed up to geographically separate locations.

DCS-17: Datacenter Metrics

Control Specification

Establish, monitor and report data center security metrics to secure data center assets and services.

Auditing Guidelines for Model Providers (MP)

1. Verify that monitoring is in place for infrastructure supporting model training, inference, and storage of model artifacts.

2. Confirm that metrics include compute health, storage reliability, access events, and configuration deviations.

3. Verify that deviations from approved environments are investigated and documented.

4. Confirm that monitoring supports continuity of model development and deployment operations.

DCS-18: Datacenter Operations Resilience

Control Specification

Define, implement and evaluate processes, procedures and technical measures to ensure continuous operations.

Auditing Guidelines for Model Providers (MP)

1. Verify that recovery capabilities exist for environments supporting model training, inference, and model artifact storage.

2. Confirm that training pipelines and inference services can be resumed or restored following disruptions.

3. Verify that backups of model artifacts and training data are maintained and tested.

4. Confirm that continuity planning includes dependencies on orchestration platforms and infrastructure services.

DSP: Data Security and Privacy Lifecycle Management

DSP-01: Security and Privacy Policy and Procedures

Control Specification

Establish, document, approve, communicate, apply, evaluate and maintain policies and procedures for the preparation, classification, protection and handling of data throughout its lifecycle, and according to all applicable laws and regulations, standards, and risk level. Review and update the policies and procedures at least annually, or upon significant changes.

Auditing Guidelines for Model Providers (MP)

1. Examine the MP’s policy and procedures related to data security and privacy.

2. Determine if a framework exists to ensure that the MP monitors the regulatory and legislative environment for changes applicable to the MP’s data security and privacy policy and procedures. Confirm whether the MP has documented the roles and responsibilities that support its policy management.

3. Confirm whether the data security and privacy policy addresses the requirement that the MP’s data is used only for authorized purposes and in compliance with legislation and regulation.

4. Examine if the security and privacy policy and procedures are reviewed and updated annually.

5. Examine documentation to determine if the function responsible for data security and privacy compliance reviews the information to determine whether the organization complies with current legislation and regulations.

6. Determine if the organization has a process for approving and communicating the classification, protection, preparation, and handling of data throughout its lifecycle.

7. Verify that policies specifically address unique AI-related considerations such as bias in training data, data provenance, and special categories of data.

DSP-02: Secure Disposal

Control Specification

Apply industry accepted methods for the secure disposal of data from storage media such that data is not recoverable by any forensic means.

Auditing Guidelines for Model Providers (MP)

1. Examine the MP’s procedures and technical requirements related to the secure disposal of data from storage media. Establish that this process and key controls comply with the MP’s data privacy and security policy. Establish whether the MP has documented the roles and responsibilities for this process.

2. Select a sample of disposal requests (if available) and assess whether they have followed the process through to completion. Confirm that all evidence was formally documented and recorded.

3. Examine measure(s) that evaluate(s) this process and determine if the measure(s) address(es) implementation of the process/control requirement(s) as stipulated.

4. Obtain and examine supporting documentation maintained as evidence of these metrics to determine if the office or individual responsible reviews the information and if identified issues were investigated and corrected. Examine related records to determine if the individual or office conducted any follow-ups on the deviations to verify they were corrected as intended.

5. Verify that industry-accepted methods for secure data disposal are defined and implemented, ensuring data is not recoverable by any forensic means.

6. Verify that data disposal techniques include secure deletion, overwriting, and physical destruction of storage media.

7. Verify compliance with relevant data protection laws and organizational policies throughout the data disposal process.

8. Verify the effectiveness of technical measures such as certified data wiping tools and secure destruction methods.

9. Verify that disposal methods address different types of data storage, including distributed file systems, object storage, and specialized ML platforms.

10. Review evidence of implementation, including logs or other documentation that confirms the proper erasure of sensitive training data when it is no longer needed.

11. Assess whether disposal procedures include verification steps to confirm complete data removal, particularly from distributed systems where data may be replicated.

12. Verify that secure disposal requirements are incorporated into the model development lifecycle and data management workflows.

13. Examine how the MP addresses secure disposal of model versions, experiment data, and associated metadata that may contain sensitive information.

DSP-03: Data Inventory

Control Specification

Create and maintain a data inventory, at least for any sensitive, regulated and personal data. Review and update the inventory at least annually, or upon significant changes.

Auditing Guidelines for Model Providers (MP)

1. Examine the MP’s procedures and technical requirements for the population and management of its data inventory. Establish that this process and key controls comply with the MP’s data privacy and security policy. Establish whether the MP has documented the roles and responsibilities for this process.

2. Select a sample of entries to ensure they have been recorded correctly on the inventory. The sample must include a proportion of sensitive and personal data entries.

3. Assess whether data inventory management meets the MP’s expectations from the defined procedures and technical requirements.

4. Examine measure(s) that evaluate(s) this process and determine if the measure(s) address(es) implementation of the process/control requirement(s) as stipulated.

5. Verify that a comprehensive data inventory is created, including all sensitive and personal data.

6. Verify that data sources, types, usage, and ownership are identified and documented.

7. Verify that the data inventory is maintained and updated regularly to reflect changes in data assets and processing activities.

8. Verify compliance with relevant data protection laws (e.g., GDPR, CCPA) and organizational policies throughout the data inventory process.

9. Examine the MP’s documented inventory of training datasets, model artifacts, and intermediate data that contain sensitive or personal information, verifying completeness across all AI development environments.

10. Verify that the inventory includes critical metadata for each dataset, including data types, sensitivity classifications, sources, processing purposes, retention periods, and protection measures.

11. Review processes for identifying and cataloging sensitive or personal information within training data, including automated scanning, manual review procedures, or sampling methodologies.

12. Assess how the inventory is maintained as datasets evolve, particularly when new data is incorporated or datasets are modified during the model development lifecycle.

13. Verify that the inventory enables traceability of sensitive data from the source through various training and testing phases to model deployment.

14. Review evidence that the inventory is regularly validated for accuracy and completeness, with documented updates reflecting current data assets.

DSP-04: Data Classification

Control Specification

Classify data according to its type, criticality and sensitivity level.

Auditing Guidelines for Model Providers (MP)

1. Examine the MP’s policy, procedures, and technical requirements for classifying data. Establish that this process and key controls comply with the MP’s data privacy and security policy. Establish whether the MP has documented the roles and responsibilities for this process.

2. Establish if the MP’s data classification matrix is aligned with the MP’s data classification requirements in terms of data type, criticality and sensitivity level.

3. Select a sample of data to confirm that each item has been classified appropriately.

4. Examine the measure (s) that evaluate this process and determine if they address the implementation of the process/control requirement (s) as stipulated. Verify that technical measures such as labeling, tagging, and access controls are used to enforce data classification.

5. Verify that data classification criteria are based on the MP’s specific needs and regulatory requirements.

6. Verify that data classification processes include regular reviews and updates to reflect data types, criticality and sensitivity levels changes.

7. Verify that all training datasets are assessed and classified according to the defined framework, with documented classifications for each dataset.

8. Review mechanisms for classifying derived data and model artifacts, ensuring they inherit appropriate classifications from source training data.

9. Assess how classification metadata is maintained throughout data transformations in the model development pipeline.

10. Verify that protection controls (access restrictions, encryption, monitoring) are implemented based on classification levels and consistently applied across development environments.

11. Review procedures for handling data with mixed classification levels within datasets, including techniques for segmentation or applying the highest applicable level.

DSP-05: Data Flow Documentation

Control Specification

Create data flow documentation to identify what data is processed, stored or transmitted where. Review data flow documentation at defined intervals, at least annually, and upon significant changes.

Auditing Guidelines for Model Providers (MP)

1. Examine the MP’s procedures and technical requirements for data flow documentation, and ensure that a review is carried out at least annually and after any change. Establish that this process and key controls comply with the MP’s data privacy and security policy. Establish whether the MP has documented the roles and responsibilities for this process.

2. Select a sample of documents to check that they have been completed to the correct specifications and reviewed.

3. Review whether data flow documentation includes an assessment of the accuracy, completeness, timeliness, and sustainability of the data (flow).

4. Identify if data flow documentation includes how data is processed, stored, and transmitted.

5. Verify that data flow documentation is reviewed at defined intervals, at least annually, and after any significant changes to the data processing environment.

6. Verify compliance with relevant data protection laws and organizational policies throughout the data flow documentation process.

7. Verify that documentation identifies what types of data (including sensitive or personal data) are processed at each stage, where data is stored during different phases, and how data moves between development environments.

8. Review documentation of data transformations and derivations during model development, confirming that it accounts for how data characteristics change throughout the pipeline.

9. Assess whether documentation identifies ephemeral versus persistent data stores and specifies retention periods for data at various stages.

10. Verify that documentation is reviewed at defined intervals (at least annually) and updated whenever changes occur to data sources, processing pipelines, or model architectures.

11. Examine specific examples of documentation updates following significant changes, such as new training data sources, modified preprocessing techniques, or revised model architectures.

DSP-06: Data Ownership and Stewardship

Control Specification

Document ownership and stewardship of all relevant documented personal and sensitive data. Perform review at least annually.

Auditing Guidelines for Model Providers (MP)

1. Examine the MP’s personal and sensitive data ownership and stewardship process. Determine if the documentation defines roles and responsibilities. Establish that this process and key controls comply with the MP’s data privacy and security policy. Establish whether the organization has documented the roles and responsibilities for this process.

2. Establish that the MP maintains a source(s) of record of data owners and stewards and the records for which they are responsible. This must include personal and sensitive data.

3. In the absence of a documented procedure, interview the control owner(s) responsible for key staff involved in the process and/or other relevant stakeholders impacted by the process/control requirement(s) and determine if the requirement(s) is/are understood. Evidence may be provided by observing individuals, systems, and/or processes associated with data management to determine if the process requirements are generally understood and implemented consistently.

4. Examine if the documentation is reviewed on an annual basis.

5. Verify that a data responsibility matrix detailing data types, associated obligations, and responsible persons or roles has been created.

6. Verify that the MP maintains a source of record for data owners and the records for which they are responsible.

7. Verify that the documentation clearly defines roles and responsibilities for different personnel (data scientists, engineers, compliance staff) who handle sensitive training data throughout the model development lifecycle.

8. Review processes for tracking data ownership as datasets are transformed, combined, or derived during model development, ensuring clear stewardship documentation at each stage.

9. Assess documentation of third-party data relationships, including ownership agreements, licensing terms, usage restrictions, and stewardship obligations for external datasets.

10. Verify that the documentation addresses stewardship responsibilities for model artifacts that may contain or expose elements of the original training data.

11. Confirm that ownership and stewardship documentation is reviewed at least annually and updated when data sources, development processes, or personnel roles change.

DSP-07: Data Protection by Design and Default

Control Specification

Develop systems, products, and business practices based upon a principle of security by design and industry best practices.

Auditing Guidelines for Model Providers (MP)

1. Examine whether the MP’s policy, standards, and procedures create a framework that fosters a culture and expectation of data protection by design and default.

2. Establish whether the MP has documented the roles and responsibilities involved.

3. Review the MP’s data breaches log, security incidents log, and project change failure records for examples of this requirement not being followed correctly. Further, confirm that action plans were identified and carried out.

4. Verify that security controls are embedded at every stage of the system development lifecycle.

5. Verify the effectiveness of technical measures such as secure coding practices, encryption, and access controls.

6. Verify that regular assessments and audits are conducted to evaluate the effectiveness of security measures and identify potential risks.

7. Verify that all processes, procedures, and technical measures related to security by design are thoroughly documented and regularly updated to reflect changes in industry best practices and regulations.

8. Examine model development methodology documentation to verify that security and data protection requirements are integrated into each phase of the model lifecycle, from data collection through deployment.

9. Verify that secure development practices, including code review, vulnerability scanning, and secure coding standards, are applied to all code used in data preprocessing, feature engineering, model training, and validation.

10. Review data protection mechanisms implemented in training environments, confirming they include access controls, encryption, anonymization or minimization techniques, and secure disposal procedures.

11. Assess how privacy-preserving techniques (e.g., differential privacy, federated learning) are evaluated and incorporated when appropriate to enhance data protection in model development.

12. Evaluate how the MP’s model architecture design process accounts for security considerations, including resilience to adversarial attacks, prevention of data leakage, and secure inference.

13. Verify that security testing (e.g., red teaming, adversarial testing) is conducted during model development rather than only at the end of the process.

DSP-08: Data Privacy by Design and Default

Control Specification

Develop systems, products, and business practices based upon a principle of privacy by design and industry best practices. Ensure that systems’ privacy settings are configured by default, according to all applicable laws and regulations.

Auditing Guidelines for Model Providers (MP)

1. Examine whether the MP’s policy, standards, and procedures create a framework that fosters a culture and expectation of privacy by design. Determine whether this content addresses the directive of the MP’s culture and whether practices reflect privacy by design and industry best practices.

2. Examine whether the MP’s governance framework, documents, controls, and metrics satisfy the organization, and if its sub-processors comply with this requirement. Establish whether the MP has documented the roles and responsibilities involved.

3. Obtain and examine supporting documentation maintained as evidence of these metrics to determine if the office or individual responsible reviews the information, and if identified issues were investigated and remediated appropriately.

4. Obtain evidence of the systems’ privacy settings and the laws and regulations that apply to the MP. Determine if the configurations are implemented as defined by the applicable laws and regulations.

5. Verify that processes, systems, and applications used for the collection and processing (including use, disclosure, retention, transmission, and disposal) are limited to what is necessary for the identified purpose.

6. Verify that the MP limits data collection to the minimum necessary for the identified purposes.

7. Verify that the MP limits the data processing to what is accurate, adequate, relevant, and necessary for the identified purposes.

8. Verify that the MP defines and documents data minimization objectives and uses mechanisms (such as de-identification) to meet those objectives.

9. Verify that the MP either deletes or renders data in a form that does not permit identification when it no longer requires access to identifiable forms of data unless there is a legal requirement or business justification to retain it in identifiable form.

10. Verify that the MP ensures that temporary files created during data processing are deleted (e.g., erased or destroyed) following documented procedures within a specified, documented time frame.

11. Verify that the MP does not retain data for longer than necessary for the purposes for which it was processed.

12. Verify that the MP follows documented policies, procedures, and/or mechanisms when disposing of data.

13. Verify that the MP subjects data (e.g., sent to another organization) over a data-transmission network to appropriate controls to ensure data reaches its intended destination.

14. Verify the implementation of data minimization practices, confirming that training data collection and processing are limited to what is necessary for model development and exclude excessive personal information by default.

15. Review privacy-enhancing techniques applied to training data, such as anonymization, pseudonymization, or differential privacy, confirming that they are default rather than optional measures, and implement them.

16. Assess mechanisms to prevent and detect model memorization of sensitive personal data, including privacy testing of model outputs to identify potential data leakage.

17. Verify that model documentation and APIs communicate privacy implications, data usage limitations, and appropriate use cases to downstream consumers by default.

18. Examine how regulatory requirements from different jurisdictions are incorporated into default privacy settings for models, ensuring compliance with the most restrictive applicable regulations.

DSP-09: Data Protection Impact Assessment

Control Specification

Conduct a Data Protection Impact Assessment (DPIA) to evaluate the origin, nature, particularity and severity of the risks upon the processing of personal data, according to any applicable laws, regulations and industry best practices.

Auditing Guidelines for Model Providers (MP)

1. Examine procedures related to DPIA risk assessment and determine whether, once a requirement has been established, the MP identifies and grades the associated risks, reports, and prioritizes the remediation of risks and non-compliance activities.

2. Examine whether the DPIA process and templates align with the MP’s risk methodology and taxonomy.

3. Determine if the risks’ origin, nature, particularity, and severity are evaluated according to the applicable laws, regulations, and industry best practices for the MP.

4. Establish whether the MP has documented the roles and responsibilities for this process.

5. Select a sample of DPIAs and examine evidence to confirm that each assessment was performed to identify associated risks. Further, verify that any action plans were determined and carried out appropriately. Confirm that all relevant evidence was formally documented.

6. Verify that AI systems used in PII processing are included in the DPIA evaluation process.

7. Verify identification and assessment of risks specific to AI systems, such as bias, transparency, and accountability.

8. Verify that the DPIA includes evaluating profiling based on AI systems’ data.

9. Verify that records inform the DPIA process for AI systems and are kept up-to-date.

10. Verify that DPIAs evaluate risks, including model memorization, inference attacks, and unintended data leakage, appropriate to the scale and sensitivity of the data processed.

11. Assess whether the MP’s DPIAs comply with relevant regulatory frameworks (e.g., GDPR Article 35, if applicable) and industry standards.

12. Assess whether the MP provides sufficient DPIA-relevant information to downstream consumers of their models.

DSP-10: Sensitive Data Transfer

Control Specification

Define, implement and evaluate processes, procedures and technical measures that ensure any transfer of personal or sensitive data is protected from unauthorized access and only processed within scope as permitted by the respective laws and regulations.

Auditing Guidelines for Model Providers (MP)

1. Examine the MP’s procedures and technical requirements for securing and legally transferring personal and sensitive data. Establish that this process and key controls comply with the MP’s data privacy and security policy.

2. Establish whether the MP has documented the roles and responsibilities for this process.

3. Select a range of personal and sensitive data transfers to confirm that each transfer adhered to the MP’s policy, procedures, and controls. Confirm that all relevant evidence was formally documented.

4. Obtain a sample of the technical measures implemented by the MP to determine if those measures adhere to the MP’s data privacy and security policy.

5. Verify that data transfers are protected from unauthorized access using encryption, secure communication channels, and access controls.

6. Verify compliance with relevant data protection laws (e.g., GDPR, CCPA) and organizational policies throughout the data transfer and processing activities.

7. Verify that regular assessments and audits are conducted to evaluate the effectiveness of data transfer and processing measures and identify potential risks.

8. Verify that all processes, procedures, and technical measures related to data transfer and processing are thoroughly documented and regularly updated to reflect changes in laws and regulations.

9. Verify implementation of data transfer scope limitations that prevent processing beyond authorized purposes during model training, tuning, and inference.

10. Review cross-border transfer compliance mechanisms (e.g., SCCs, adequacy decisions) for model training and operation data.

11. Assess documentation of data flow patterns, identifying all points where personal data is transferred during model development and deployment.

12. Verify encryption implementation for data transfers during all model-related activities.

13. Evaluate technical controls preventing unauthorized extraction of sensitive data from models.

14. Review data transfer monitoring and logging specific to model development and operation processes.

DSP-11: Personal Data Access, Reversal, Rectification and Deletion

Control Specification

Define and implement, processes, procedures and technical measures to enable data subjects to request access to, modification, or deletion of their personal data, according to any applicable laws and regulations.

Auditing Guidelines for Model Providers (MP)

1. Examine whether the MP’s policy and procedures related to data privacy address the requirement that authorized users must be able to access, modify, or delete personal data, and whether it is handled according to the applicable laws and regulations.

2. Establish whether the MP has processes to manage and respond to data access requests from data subjects and whether it has documented the roles and responsibilities for this process.

3. Select a range of data changes to confirm that only authorized users can access, modify, and delete personal data successfully. Select a sample of data access requests to establish that these were completed correctly following the MP’s processes. Confirm that all relevant evidence was formally documented.

4. Verify that data subjects are informed about their rights and the procedures to exercise them.

5. Verify implementation of technical capabilities to identify and extract personal data used in model training when required for subject access requests.

6. Assess processes for removing specific personal data from training datasets and, where technically feasible, from trained models upon request.

7. Review mechanisms to prevent the regeneration of deleted personal data in model outputs.

8. Examine documentation of technical limitations in fulfilling data subject rights for specific AI technologies, including implemented alternative measures.

9. Verify that data lineage tracking can identify all models potentially containing a specific individual’s data.

10. Assess whether testing verifies the effectiveness of data subject request fulfillment across model operations.

DSP-12: Limitation of Purpose in Personal Data Processing

Control Specification

Define, implement and evaluate processes, procedures and technical measures to ensure that personal data is processed according to any applicable laws and regulations and for the purposes declared to the data subject.

Auditing Guidelines for Model Providers (MP)

1. Examine whether the MP’s policy and procedures related to data privacy address the requirement that the data the MP is responsible for is processed lawfully and used only for the purposes stated to data subjects.

2. Establish whether the MP has documented the roles and responsibilities for this process.

3. Review the MP’s data breaches and confirm that action plans were identified and carried out appropriately. Confirm that all supporting evidence was formally documented.

4. Review the MP’s processes that inform data subjects why it requests this data and what it will be used for. Confirm that any MP documentation (including web page content) is subject to formal periodic review for relevance and compliance with legislation and regulation.

5. Review the technical measures implemented to ensure that personal data is processed according to applicable laws and regulations.

6. Verify that the purposes for processing personal data are declared and documented to the data subject.

7. Verify the effectiveness of technical measures such as encryption, access controls, and data anonymization used during data processing.

8. Verify that all processes, procedures, and technical measures related to data processing are thoroughly documented and regularly updated to reflect changes in laws and regulations.

9. Verify implementation of controls that enforce purpose limitations during model training, validation, and inference.

10. Review records mapping specific datasets to their approved processing purposes.

11. Assess mechanisms that prevent model outputs from exceeding declared processing purposes.

12. Evaluate processes for validating that model behavior adheres to specified purpose boundaries.

13. Verify implementation of technical measures preventing unauthorized repurposing of personal data during model development.

14. Review documentation of purpose compatibility assessments conducted before using existing data for new AI applications.

DSP-13: Personal Data Sub-processing

Control Specification

Define, implement and evaluate processes, procedures and technical measures for the transfer and sub-processing of personal data within the service supply chain, according to any applicable laws and regulations.

Auditing Guidelines for Model Providers (MP)

1. Examine the MP’s contractual terms, procedures, roles, responsibilities, documents, and technical measures for transferring personal data and sensitive data to subprocessors and how subprocessors are to treat this data.

2. Establish whether the MP has documented the roles and responsibilities for this process.

3. Select a sample of data transfers to subprocessors to establish that the controls and reporting of the subprocessors comply with the organization’s data privacy and security policy.

4. Examine the MP’s contractual requirements for subprocessor compliance, reporting, and non-compliance sanctions and the MP’s right to audit. Establish subprocessors’ processes, controls, and metrics to comply with the organization’s requirements.

5. Verify that contracts with suppliers and sub-processors include clauses that comply with applicable laws and regulations regarding the transfer and sub-processing of personal data.

6. Verify the effectiveness of technical measures such as encryption, secure communication channels, and data masking used during data transfer and sub-processing.

7. Verify that regular assessments and audits are conducted to evaluate the effectiveness of data transfer and sub-processing measures and identify potential risks.

8. Verify that all processes, procedures, and technical measures related to data transfer and sub-processing are thoroughly documented and regularly updated to reflect changes in laws and regulations.

9. Verify the existence and adequacy of a complete sub-processor inventory, including the purpose of processing, categories of data transferred, and data flow documentation.

10. Review data transfer mechanisms (e.g., SCCs, BCRs) implemented to ensure lawful transfers of personal data to sub-processors across jurisdictions.

11. Assess the MP’s process for conducting due diligence on sub-processors, including security assessments and compliance verification.

12. Verify that appropriate contractual safeguards (e.g., DPAs, contractual clauses) are in place with all personal data subprocessors.

13. Examine processes for notifying customers/data controllers about new or changed sub-processors, including approval mechanisms.

14. Review evidence of periodic sub-processor compliance assessments against regulatory requirements and contractual obligations.

DSP-14: Disclosure of Data Sub-processors

Control Specification

Define, implement and evaluate processes, procedures and technical measures to disclose the details of any personal or sensitive data access by sub-processors to the data owner prior to initiation of that processing.

Auditing Guidelines for Model Providers (MP)

1. Policies, Roles, and Contracts: Examine the MP’s documented policies, procedures, and contractual requirements mandating disclosure of sub-processors to customers before processing begins. Verify that roles and responsibilities for managing and communicating disclosures are documented and assigned. Review contracts with sub-processors and customers to confirm they include: provisions for compliant and ethical handling of PII, disclosure of subcontractors, requirements for sub-processors to meet equivalent standards, and data minimization clauses limiting disclosure scope.

2. Sample-Based Validation: Select a sample of sub-processor data flows and validate that disclosures were made prior to processing and controls and reporting align with MP’s privacy/security policies.

3. Disclosure Records and Record-Keeping: Verify the MP maintains a comprehensive sub-processor inventory, detailing what personal/sensitive data each sub-processor accesses and for what purpose. Check that records of disclosures include: what was disclosed, when, to whom, and the authority/legal basis. Ensure the record-keeping system supports traceability of approvals and disclosures over the data/model lifecycle (training, deployment, maintenance).

4. Customer Notification and Legal Requests: Confirm the MP has a documented process to: notify customers of legally binding PII disclosure requests, reject non-legally binding requests unless the customer consents, and ensure timely notifications consistent with legal and contractual obligations.

5. Customer-Facing Disclosure Process: Review templates and records of sub-processor disclosure notifications to customers, verifying that they include: identity, location, and purpose of processing; categories of data accessed; and security measures in place. Assess the MP’s process for obtaining and documenting customer approvals before engaging new sub-processors or making significant changes. Verify mechanisms ensure disclosures are made before processing and allow sufficient time for customer review.

DSP-15: Limitation of Production Data Use

Control Specification

Obtain authorization from data owners, and manage associated risk before replicating or using production data in non-production environments.

Auditing Guidelines for Model Providers (MP)

1. Verify that the procedures and technical requirements for using production data (e.g., real-world datasets or customer-sourced data) in model training, fine-tuning, or evaluation environments are defined.

2. Verify if approvals are obtained for using production-like data (e.g., customer-contributed content) in training or research environments.

3. Verify if the data is anonymized, obfuscated, and securely deployed in development and training pipelines.

4. Verify if deviations from approved data usage protocols in training environments have been documented and approved.

5. Verify if data handling and processing procedures are reviewed regularly and updated to reflect evolving regulations and advances in AI model training practices.

6. Verify if training programs exist to ensure engineering and research staff understand protocols for handling sensitive or production data in non-production environments.

DSP-16: Data Retention and Deletion

Control Specification

Data retention, archiving and deletion is managed in accordance with business requirements, applicable laws and regulations.

Auditing Guidelines for Model Providers (MP)

1. Verify if policies and procedures are defined for retention, archiving, and deletion of training data, datasets, and related artifacts, including assignment of roles and responsibilities.

2. Verify if data types (e.g., text corpora, images), data sources, ownership, and retention periods are documented in a retention schedule aligned with legal and licensing requirements.

3. Verify if model training datasets and related data records are retained in accordance with the established retention schedule.

4. Verify if contractual agreements with third-party data providers include retention and deletion obligations.

5. Verify if datasets are deleted using industry-standard methods after the end of their permitted use.

6. Verify if logs of data deletion and archiving (e.g., dataset lifecycle logs) are maintained, monitored, and reviewed.

7. Verify if access controls and protections are in place to prevent data leaks or unauthorized access during retention.

8. Verify if data retention policies are reviewed and updated to reflect evolving AI regulations and data governance requirements.

9. Verify if policies address the retention and lifecycle management of data used in training or evaluating AI models.

10. Verify if model architectures and training pipelines are designed to minimize long-term retention of sensitive data such as PII.

11. Verify if training datasets are anonymized or de-identified wherever possible, especially for sensitive data.

12. Verify if access to training data is governed by strict controls and access management systems.

13. Verify if training data is securely deleted when no longer needed or when the retention period expires.

14. Verify if internal audits or tooling are in place to monitor compliance with data retention policies related to model development.

DSP-17: Sensitive Data Protection

Control Specification

Define and implement, processes, procedures and technical measures to protect sensitive data throughout its lifecycle.

Auditing Guidelines for Model Providers (MP)

1. Verify whether the organization’s policy and procedures related to data privacy include guidelines for managing and protecting sensitive data used in model training, fine-tuning, evaluation, and inferencing throughout its lifecycle.

2. Verify whether the organization has documented roles and responsibilities for managing privacy risks associated with AI model development and deployment.

3. Verify that data classification policies are clear and aligned with training data sensitivity; ensure appropriate anonymization and consent procedures are followed; inspect compliance with applicable data protection regulations; validate technical safeguards during data ingestion and model training; interview ML engineers and privacy leads; and confirm that privacy controls are regularly updated.

4. Verify that systems, processes, and controls are in place to protect sensitive training or user-contributed data throughout its lifecycle, including ingestion, training, storage, and deployment.

5. Verify the organization’s records of data privacy incidents involving model development (e.g., leakage of training data in outputs); confirm that any remediation plans were executed and documented appropriately.

6. Verify that AI risk management strategies include safeguards to mitigate reidentification risks, exposure of sensitive data, and bias related to protected attributes in training data.

7. Verify that the organization has a defined process for handling incidents involving AI models, including the reporting, investigation, mitigation, and documentation of privacy-related events.

DSP-18: Disclosure Notification

Control Specification

The service providers must implement and describe to service customers the procedure to manage and respond to requests for disclosure of Personal Data by Law Enforcement Authorities according to applicable laws and regulations.

Auditing Guidelines for Model Providers (MP)

1. Verify if procedures and technical requirements address how personal data requests from law enforcement related to training datasets or model outputs will be handled.

2. Verify if procedures for responding to law enforcement requests comply with internal privacy and security policies and applicable data protection regulations.

3. Verify if roles and responsibilities are defined for authorizing and managing disclosure of data or model outputs to authorities.

4. Verify if disclosure notifications follow established approval workflows and use secure communication channels.

5. Verify that all actions, approvals, and correspondence related to disclosures are formally documented.

6. Verify that disclosure notifications are issued within required legal and regulatory timeframes.

7. Verify if procedures are regularly reviewed and updated to align with changing legal obligations or model capabilities.

8. Verify if staff involved in disclosure processes receive training on legal obligations and internal procedures.

9. Verify if mechanisms are in place for logging and tracking data requests, including legal basis and outcomes.

10. Verify if processes exist to report and address deviations or non-compliance in disclosure handling.

11. Verify if specific requirements are defined for handling disclosures that involve AI-generated outputs or model-inferred data.

12. Verify if access to AI-generated data during law enforcement disclosures is protected by strict access controls.

13. Verify if logging and auditing mechanisms exist for monitoring disclosure requests related to AI model data.

DSP-19: Data Location

Control Specification

Define and implement, processes, procedures and technical measures to specify and document the physical locations of data, including any locations in which data is processed or backed up.

Auditing Guidelines for Model Providers (MP)

1. Verify the organization’s policies, procedures, technical requirements, and documentation address the physical storage locations of training and model data, and the ethical use of AI in managing such data storage and processing.

2. Verify whether roles and responsibilities related to managing AI model storage infrastructure and data governance are clearly documented.

3. Verify that policies include geographic and jurisdictional guidelines governing data storage and processing used in model training and deployment.

4. Verify that the organization maintains authoritative records of physical storage locations used for model data and training datasets, ensuring traceability of data lineage.

5. Verify that records of data storage locations are accurate, complete, and comply with stated policies.

6. Verify whether roles and obligations regarding AI storage systems are documented both internally and in agreements with suppliers or cloud providers.

7. Verify that AI models and associated data storage systems comply with organizational policies regarding security, ethical use, and regulatory requirements.

8. Verify that monitoring and auditing procedures exist to ensure AI model storage systems meet ethical standards and compliance requirements.

9. Verify risk management strategies are in place to address ethical considerations, bias mitigation, and transparency within AI data storage and model training pipelines.

10. Verify that incident management processes cover AI model storage-related events, including reporting, investigation, and remediation.

DSP-20: Data Provenance and Transparency

Control Specification

Define, implement and evaluate processes, procedures and technical measures to: 1) Document and trace data sources, and 2) Make the data source available according to legal and regulatory requirements

Auditing Guidelines for Model Providers (MP)

1. Verify that all data sources used for training and fine-tuning models (including origin, type, and format) are clearly identified and documented.

2. Verify that data lineage documentation illustrates the flow of training data from source to preprocessing and model ingestion.

3. Verify that a data dictionary is maintained describing datasets, features, labels, and their relationships used in model training.

4. Verify that provenance records capture all transformations and curation steps applied to training data.

5. Verify that automated systems monitor access to and changes in training datasets and data pipelines.

6. Verify that tracking mechanisms are in place to ensure the integrity of datasets during collection, transformation, and use.

7. Verify that processes address challenges related to dataset scalability, privacy risks (e.g., PII), and high-dimensional data complexity.

8. Verify compliance with data protection regulations (e.g., GDPR, CCPA) and industry-specific standards when sourcing or processing data.

9. Verify that access controls and encryption are applied to datasets used for model development and training.

10. Verify that data retention policies are enforced and datasets are securely deleted after their retention period.

11. Verify that teams are regularly trained on secure data handling, documentation practices, and regulatory compliance for training data.

12. Verify that logs and metadata related to dataset origin, processing, and transformation are retained and auditable.

13. Verify that version control systems are in place to track updates to training datasets and resulting model versions.

14. Verify that procedures for disclosing training data or metadata to stakeholders or authorities are defined, documented, and legally compliant.

DSP-21: Data Poisoning Prevention & Detection

Control Specification

Define, implement and evaluate processes, procedures and technical measures to prevent data poisoning in AI models and continuously detect such.

Auditing Guidelines for Model Providers (MP)

1. Verify that data sources used for training and fine-tuning models are validated to prevent malicious or poisoned data from entering the system.

2. Verify that data quality checks are systematically performed to detect and remove corrupted, biased, or suspicious data before training.

3. Verify that automated monitoring systems are implemented to identify unusual patterns or anomalies in training datasets.

4. Verify that training techniques such as adversarial training or data sanitization methods are applied to improve model resilience to poisoned data.

5. Verify that strict access controls restrict unauthorized modification or insertion of training datasets.

6. Verify that data encryption is implemented to protect training data in storage and transit from unauthorized access or tampering.

7. Verify monitoring systems are in place to detect data tampering or poisoning attempts during data collection, storage, and training phases.

8. Verify that incident response plans specifically address potential data poisoning events, including detection, investigation, and remediation.

9. Verify that employees involved in data collection, processing, and model training are trained to recognize and report data poisoning threats.

10. Verify the use of automated tools to continuously monitor data integrity and detect anomalies during the training data lifecycle.

DSP-22: Privacy Enhancing Technologies

Control Specification

Use Privacy Enhancing Technologies for training data, informed by risk and privacy impact analysis and business use cases.

Auditing Guidelines for Model Providers (MP)

1. Verify business use cases for implementing PETs (e.g., differential privacy, federated learning) during model training are clearly defined and documented.

2. Verify that PET implementations in model training pipelines are continuously monitored and evaluated for effectiveness and risk mitigation.

3. Verify that PETs used in model development comply with relevant privacy laws and AI-specific standards.

4. Verify that metrics and KPIs (e.g., privacy loss budget, epsilon value) are defined and tracked to measure PET effectiveness during training.

5. Verify that ML engineers and data scientists receive training on integrating, monitoring, and auditing PETs in the model lifecycle.

6. Verify that PET libraries and components are regularly updated with security patches.

7. Verify that logs from PET usage (e.g., federated learning rounds, privacy budget consumption) are reviewed and monitored for anomalies or misuse.

8. Verify that independent third-party security and privacy assessments are periodically performed on PET-enabled model training environments.

DSP-23: Data Integrity Check

Control Specification

Regularly validate the consistency and conformity of training, fine-tuning or augmentation data. Implement dataset versioning to ensure traceability and enforce restrictions to prevent unauthorized changes.

Auditing Guidelines for Model Providers (MP)

1. Verify that all data sources used in model training and fine-tuning are clearly identified and fully traceable.

2. Verify that systems are implemented to track and log any changes or updates to datasets used in model development.

3. Verify that automated tools are utilized to continuously monitor data integrity and detect anomalies in training data.

4. Verify that access controls prevent unauthorized modifications to training data.

5. Verify that sensitive training data is encrypted to prevent unauthorized access and tampering.

6. Verify that version control systems track changes to datasets and models throughout the development lifecycle.

7. Verify that employees involved in data handling and model training are trained on best practices for data integrity.

8. Verify that procedures exist and are followed to address any data integrity issues that arise during model development.

DSP-24: Data Differentiation and Relevance

Control Specification

Ensure training-data differentiation and relevance to the intended use of the AI Model.

Auditing Guidelines for Model Providers (MP)

1. Verify if the model provider has a documented process for data acquisition, labeling, augmentation, and balancing, including version control or change logs showing a robust and continuous approach.

2. Verify if the model provider has implemented procedures to measure data quality, accuracy, diversity, and completeness, and has a process to evaluate the success and fairness of the training dataset.

3. Verify if the AI model’s design and training process for potential biases is reviewed by domain experts who validate the relevance and appropriateness of the data.

4. Verify if metrics such as demographic parity, equalized odds, or other fairness indicators are used to detect bias in the model and data.

5. Verify if the model provider has identified and addressed any performance disparities between different demographic or user groups.

6. Verify if the model provider applies bias detection methods, including statistical tests and scenario simulations, to detect and understand the impact of biases.

7. Verify that the model provider has comprehensive documentation of the full AI lifecycle, including data sources, model design decisions, and decision-making processes.

8. Verify if clear and accessible information about the AI system’s functioning is provided to relevant stakeholders (internal teams, regulators, users).

9. Verify if continuous monitoring and improvement tools are implemented to regularly test and refine the model based on performance and fairness metrics.

10. Verify if the AI system complies with applicable privacy regulations and is updated to reflect evolving AI governance and compliance standards.

11. Verify if the model provider gathers feedback from diverse stakeholders on the AI system’s impact and relevance, using this feedback to drive adjustments and improvements.

12. Verify that the model provider maintains detailed documentation on data sources, including origin, context, and preprocessing steps, ensuring transparency in data lineage for model training.

13. Verify that the model provider uses high-quality, accurate, and regularly updated data to keep the model relevant to current trends and scenarios.

14. Verify that domain-specific data tailored to the application’s needs is used, covering various contexts and functions relevant to the industry or use case.

15. Verify that the model provider conducts regular checks to detect and mitigate biases both in data and model outputs.

16. Verify that data governance policies are strictly adhered to and relevant privacy regulations (e.g., GDPR, CCPA) are complied with throughout model development.

17. Verify that mechanisms are in place to protect sensitive information and maintain the integrity of data used in model training.

18. Verify if continuous monitoring tools track the performance and relevance of the data and model, and if feedback from end-users and stakeholders is collected and leveraged to ensure ongoing alignment with needs.

GRC: Governance, Risk and Compliance

GRC-01: Governance Program Policy and Procedures

Control Specification

Establish, document, approve, communicate, apply, evaluate and maintain policies and procedures for an information governance program, which is sponsored by the leadership of the organization and related to AI systems as well. Review and update the policies and procedures at least annually, or upon significant changes.

Auditing Guidelines for Model Providers (MP)

1. Examine whether the organization has established a comprehensive strategy for information governance, ensuring AI model governance is explicitly included, such as the governance of model lifecycle management, training data oversight, and explainability requirements, leadership sponsorship is documented, and governance policies, standards, and procedures are formally approved, communicated, and applied.

2. Confirm that policies, standards, and procedures are reviewed and updated at least annually, with documented evidence of the review process, leadership approval, and modifications made in response to evolving AI-related governance requirements or regulatory changes.

GRC-02: Risk Management Program

Control Specification

Establish and maintain a formal, documented, and leadership-sponsored AI Risk Management (AIRM) program that includes policies and procedures for identification, evaluation, ownership, treatment, and acceptance of risks.

Auditing Guidelines for Model Providers (MP)

1. Program Examination

   • a. Verify that a formal, documented AI Risk Management (AIRM) program exists and is approved by executive leadership.

   • b. Confirm that the AIRM program is governed by defined roles and responsibilities related to risks associated with AI model development, training pipelines, and associated cloud infrastructure.

2. Program Content Assessment

   • a. Verify that the AIRM program includes procedures for identifying, evaluating, assigning ownership of, treating, and accepting risks across the AI model lifecycle including those associated with data ingestion, feature engineering, training, validation, and model monitoring.

   • b. Confirm whether the AIRM framework accounts for AI model-specific risks such as model bias, drift, overfitting, data leakage, adversarial manipulation, and training data integrity. If not present, assess whether a documented rationale and alternative mitigation strategy exists.

   • c. Verify that the AIRM program includes documented consideration of risks arising from use of cloud-based services supporting model operations (e.g., compute resources, storage, model hosting).

   • d. Confirm that risk ownership is clearly assigned across the AI model lifecycle, including roles for model developers, ML engineers, validators, and supporting cloud infrastructure teams.

3. Program Alignment Evaluation

   • a. Evaluate whether the AIRM program aligns with the organization’s AI model governance strategy, broader AI governance framework, and enterprise risk management strategy.

   • b. Confirm whether the AIRM framework incorporates applicable or emerging standards and regulatory expectations related to AI model management, such as model fairness, explainability, accountability, and data protection.

4. Implementation Validation

   • a. Validate implementation of the AIRM program by reviewing evidence such as model-related risk logs, assessment reports, incident trackers, or dashboards documenting model performance risks and mitigations.

   • b. Inspect a sample of model-specific risk treatment or acceptance documentation to assess the traceability, decision rationale, and evidence of risk ownership or leadership approval.

GRC-03: Organizational Policy Reviews

Control Specification

Review all relevant organizational policies and associated procedures at least annually or when a substantial change occurs within the organization.

Auditing Guidelines for Model Providers (MP)

1. Policy Examination

   • a. Verify that the organization maintains a documented list or repository of policies and associated procedures that govern AI model development, training, deployment, and related use of cloud infrastructure.

   • b. Confirm that the organization has defined criteria for determining which policies are “relevant,” particularly those addressing AI model risk, data sourcing, and MLOps.

2. Policy Assessment

   • a. Verify that policies and procedures are reviewed at least annually, with supporting documentation such as review logs, version history, and documented approvals.

   • b. Confirm that the policy review process includes designated owners (e.g., model governance leads, risk officers) and is conducted through formal governance mechanisms.

3. Review Process Evaluation

   • a. Determine whether the organization has defined what constitutes a “substantial change” in the model development environment (e.g., adoption of a new training framework, shift in model type, or regulatory updates).

   • b. Verify that the organization has a process to initiate interim policy reviews in response to these substantial changes.

4. Implementation Validation

   • a. Inspect recent policy review records to confirm that model governance–related policies are reviewed annually and reflect the current operating environment.

   • b. Examine recent changes to model development practices (e.g., modifications to model architecture, training data usage, or explainability requirements) and validate that associated policies were reviewed and updated accordingly.

GRC-04: Policy Exception Process

Control Specification

Establish and follow an approved exception process as mandated by the governance program whenever a deviation from an established policy occurs.

Auditing Guidelines for Model Providers (MP)

1. Policy Examination

   • a. Exception Process Documentation: Verify that a documented process exists for requesting, reviewing, approving, and tracking exceptions to policies related to model development, training practices, data usage, and infrastructure operations.

   • b. Governance Integration: Confirm that the exception process is defined under the broader governance program, and referenced in applicable model governance frameworks, AI risk standards, or engineering handbooks.

2. Policy Assessment

   • a. Process Criteria: Verify that the exception process includes minimum documentation requirements, roles for approval, justification rationale, and expiration or renewal procedures.

   • b. Scope and Applicability: Confirm that the exception process applies to all deviations from established model governance or infrastructure-related policies, including those that may involve use of unvetted training datasets, undocumented model changes, or deviations from bias testing protocols.

   • c. Communication Protocols: Assess whether approved exceptions are documented in a central system and communicated to relevant functions (e.g., model governance, MLOps, risk management).

3. Review Process Evaluation

   • a. Enforcement Mechanism: Determine whether tools, workflows, or internal controls are in place to prevent or detect unapproved deviations from model governance policies.

   • b. Governance Oversight: Confirm that the AI governance board, model risk committee, or equivalent body regularly reviews policy exceptions as part of oversight responsibilities.

4. Implementation Validation

   • a. Sample Testing: Review a sample of approved exceptions related to model development or infrastructure policy deviations to ensure compliance with documentation, approval, and expiration requirements.

   • b. Exception Coverage Check: Examine a sample of recent deviations (e.g., ad hoc model releases, use of nonstandard data pipelines) and verify that appropriate exceptions were formally requested and approved.

GRC-05: Information Security Program

Control Specification

Develop and implement an Information Security Program, which includes programs for all the relevant domains of the AICM.

Auditing Guidelines for Model Providers (MP)

1. Program Documentation and Scope: Verify that the organization maintains a formal, documented Information Security Program that addresses core security requirements for its systems, infrastructure, and data, particularly in relation to AI model development, training, deployment, and inference operations. Where AI activities are in scope, confirm that the program identifies and incorporates controls from relevant AICM domains (e.g., Model Lifecycle, Infrastructure, Data Protection).

2. Coverage of Model-Specific Risks and Governance: Assess whether the program includes provisions for managing risks specific to AI model development and operations, such as training data integrity, model versioning, access to model artifacts, and inference system protection. Verify that security responsibilities are clearly assigned across relevant teams (e.g., data science, MLOps, infrastructure) and that governance mechanisms (e.g., steering committees, risk oversight groups) are in place to coordinate and review model-related security concerns. Confirm that the program aligns with internal standards and relevant external frameworks for AI and cybersecurity.

3. Domain Evaluation and Program Integration: Determine whether the organization has reviewed which AICM domains are applicable to its model provider role and incorporated appropriate controls into the Information Security Program. Confirm that these controls are implemented across relevant technical and operational functions, rather than isolated within a single team.

4. Evidence of Implementation and Control Effectiveness: Review internal documentation such as security training logs, control self-assessments, risk registers, or audit reports to validate implementation of the security program. Select a sample of relevant AICM domains (e.g., Model Lifecycle, Data Management, Infrastructure) and confirm that associated policies and operational safeguards are in place and functioning as intended.

GRC-06: Governance Responsibility Model

Control Specification

Define and document roles and responsibilities for planning, implementing, operating, assessing, and improving governance programs.

Auditing Guidelines for Model Providers (MP)

1. Policy Examination

   • a. Verify that roles and responsibilities related to the governance of model development, training pipelines, deployment, and supporting infrastructure are formally documented, including responsibilities for both technical (e.g., ML engineering, MLOps) and risk-related functions (e.g., model risk management, compliance).

   • b. Confirm that governance responsibilities span all phases of the lifecycle—planning, implementation, operation, assessment, and continuous improvement—as they apply to model creation and use.

2. Policy Assessment

   • a. Assess whether governance roles include clearly defined accountability for areas such as data sourcing, model transparency, performance monitoring, and bias detection with role-specific expectations for oversight and documentation.

   • b. Confirm that governance responsibilities align with how model operations are structured within the organization (e.g., across data science, ML engineering, infrastructure, and AI risk) and that cross-functional coordination is reflected in role definitions.

3. Program Evaluation

   • a. Determine whether documented governance responsibilities are reflected in operational practices like model documentation workflows, versioning checkpoints, and approval processes and whether assigned individuals or teams are engaged in these practices on an ongoing basis.

   • b. Verify that roles include escalation paths and are tied into broader oversight or governance review mechanisms (e.g., model risk committees or technical review boards) especially for high-risk or AI production AI models.

4. Implementation Validation

   • a. Review materials such as model governance documentation, review meeting minutes, audit logs, or internal assurance results (e.g., audit reports, control self-assessments) to confirm that assigned roles are being executed.

   • b. Select a governance-related function (e.g., model validation, ethical review, infrastructure change management) and verify that responsibilities assigned in policy are reflected in actual practice, including accountability for approvals, reviews, and risk acceptance if applicable.

GRC-07: Information System Regulatory Mapping

Control Specification

Identify and document all relevant standards, regulations, legal/contractual, and statutory requirements, which are applicable to your organization. Review at least annually, or upon significant changes.

Auditing Guidelines for Model Providers (MP)

1. Policy Examination

   • a. Verify that the organization maintains a documented list of applicable regulations, standards, and contractual requirements relevant to model development, training data collection and usage, deployment, and infrastructure security.

   • b. Confirm that the inventory reflects diverse sources, including AI-specific regulations, data protection laws, open-source license obligations, export controls, and internal standards, with consideration for model inputs, outputs, and deployment environments.

2. Policy Assessment

   • a. Assess whether the inventory is reviewed and updated at regular intervals or when regulatory changes occur, and whether responsibility for maintenance is clearly assigned to accountable functions such as legal, compliance, or AI governance.

   • b. Confirm that the requirements are scoped based on how and where the models are developed, deployed, and used (e.g., geographic considerations, customer industries, types of AI models, deployment architecture).

3. Program Evaluation

   • a. Determine whether the listed obligations are integrated into model governance processes, such as data sourcing reviews, model interpretability/fairness assessments, infrastructure security controls, and deployment approvals.

   • b. Confirm that functions such as legal, AI governance, and infrastructure teams reference the documented requirements when making operational or design decisions, especially in high-risk or externally regulated AI use cases.

4. Implementation Validation

   • a. Review documentation such as model documentation checklists, dataset acquisition reviews, or policy artifacts to confirm that identified obligations are referenced during model lifecycle activities.

   • b. Select a sample of obligations (e.g., license use restrictions, AI fairness requirements, regional data protection mandates) and verify that evidence exists showing those items were considered or reflected in supporting materials.

GRC-08: Special Interest Groups

Control Specification

Establish and maintain contact with related special interest groups and other relevant entities in line with business context.

Auditing Guidelines for Model Providers (MP)

1. Policy Examination Engagement Strategy: Verify whether the organization has a documented strategy or program that encourages engagement with external groups related to AI model development, responsible AI practices, infrastructure security, or MLOps. Defined Purpose: Confirm that the rationale for participation is clearly defined (e.g., to stay current on evolving model risk, AI transparency expectations, or infrastructure compliance standards).

2. Policy Assessment Group Relevance: Assess whether participation targets relevant external communities (e.g., AI research alliances, infrastructure standards groups, ethical AI committees, GenAI governance bodies). Responsibility Assignment: Verify that participation responsibilities are assigned to relevant roles (e.g., data scientists, model risk leads, infrastructure architects) and tracked in internal governance records.

3. Program Evaluation Engagement Mechanism: Determine whether there is a process in place to select, approve, or evaluate participation in external entities based on their relevance to AI model operations and security. Information Flow: Confirm that knowledge from external collaboration is incorporated into model development practices, infrastructure policies, or internal governance briefings.

4. Implementation Validation Evidence of Participation: Review materials such as conference presentations, technical working group participation, research collaborations, or advisory board documentation to confirm active engagement. Sample Group Validation: Select a sample of external groups (e.g., NIST AI RMF working groups, MLCommons, IEEE AI standards committees) and verify that participation aligns with the model provider’s governance and business objectives.

GRC-09: Acceptable Use of the AI Service

Control Specification

Define, document and enforce policies and procedures on the acceptable use of AI services offered by the organization. Ensure effectiveness by continuous risk assessments, reviews and human oversight.

Auditing Guidelines for Model Providers (MP)

1. Examine the AI Acceptable Use Policy for adequacy, currency, and communication to relevant interested parties and users.

2. Verify that the AI Acceptable Use Policy identifies the applicable AI models and use cases subject to these guidelines.

3. Verify that the AI Acceptable Use Policy clearly defines the intended uses of the model, specifying what constitutes acceptable and prohibited use cases as applicable

4. Verify, through interviews or otherwise, that the policy is communicated to interested parties, and acknowledged as applicable. Examine policy for evidence of review by policy owner or committee at least annually.

GRC-10: AI Impact Assessment

Control Specification

Establish, document, and communicate to all relevant stakeholders an AI Impact Assessment process and its criteria to regularly evaluate the ethical, societal, operational, legal, and security impacts of the AI system throughout its lifecycle.

Auditing Guidelines for Model Providers (MP)

1. Verify that an approved AI Impact assessment process exists. Process includes methodology/criteria to evaluate foundational models aligned with global standards.

2. Verify that the evaluation process exists and is integrated into model pre-training, fine-tuning, release and post deployment monitoring.

3. Verify the model evaluation criteria and scoring mechanism exists across all the dimensions such as ethical, societal, legal, operational, and security.

4. Assess how the impact assessment methodology evaluates differential impacts across various downstream applications across all customer segments or usage patterns.

5. Verify the process to identify various stakeholders (both internal and external) and how they communicate and engage stakeholders to communicate impact assessment process, evaluation procedures, impact/risk scores and most importantly how they collect and incorporate their feedback.

GRC-11: Bias and Fairness Assessment

Control Specification

Regularly evaluate AI systems, models, datasets & algorithms for bias and fairness to ensure compliance with ethical standards.

Auditing Guidelines for Model Providers (MP)

1. Policy and Framework: Confirm a documented framework exists for evaluating bias and fairness in data, models, and outputs—aligned with Responsible AI principles and global regulations.

2. Diverse Training Data: Ensure training datasets reflect demographic, geographic, and other requirements to reduce bias.

3. Fairness in Model Design: Verify that fairness goals or constraints are incorporated during training and fine-tuning, post training

4. Bias Detection Tools: Check that validated tools and metrics are used to detect, measure, and mitigate bias throughout the model lifecycle.

5. Transparent Reporting: Confirm fairness findings, limitations, and mitigations are regularly reported through model cards or transparency reports for downstream users.

GRC-12: Ethics Committee

Control Specification

Establish an ethics committee to review AI applications, ensuring alignment with ethical standards and organizational values.

Auditing Guidelines for Model Providers (MP)

1. Verify that Model provider has a ethics committee consists of a diverse set of stakeholders needs to be involved Model development lifecycle.

2. Verify the committee roles and responsibilities are clearly defined and documented.

3. Verify that there are clear understanding of their role and have knowledge to contribute/guide towards Ethical model development.

4. Verify that there established standards for decision making and approving Models for use.

5. Verify that no individual committee member has a conflict of interest with the AI applications under review

GRC-13: Explainability Requirement

Control Specification

Establish, document, and communicate the degree of explainability needed for the AI Services.

Auditing Guidelines for Model Providers (MP)

1. Verify that the Model Provider has clearly articulated explainability requirements, informed by applicable compliance, regulatory, or ethical obligations.

2. Verify that the priority of explainability is defined and aligned with these requirements, considering the potential consequences of errors.

3. Verify that regular and structured stakeholder communication is in place to ensure explainability practices are effectively conveyed to both internal and external stakeholders.

4. Verify that there is a documented process for selecting algorithms based on explainability criteria defined in the explainability requirements.

5. Verify that the Application Provider is transparent about its explainability practices and that customers are informed and understand how decisions are made.

GRC-14: Explainability Evaluation

Control Specification

Evaluate, document, and communicate the degree of explainability of the AI Services, including possible limitations and exceptions.

Auditing Guidelines for Model Providers (MP)

1. Verify that explainability requirements are defined based on regulations, ethical principles, and expected customer compliance obligations.

2. Verify that model providers have standardized evaluation processes and frameworks (e.g., use of SHAP, LIME, feature attribution) to assess model explainability prior to release.

3. Verify that model limitations (e.g., opaque deep learning layers) and exceptions (e.g., low-risk outputs) are clearly documented and disclosed to downstream consumers.

4. Ensure that each model is accompanied by documentation (e.g., model cards) describing its degree of explainability and known constraints.

5. Verify that the explanation tools or outputs provided with the model are accessible and interpretable by non-technical users or business analysts.

GRC-15: Human supervision

Control Specification

Establish, execute, and assess processes, procedures, and technical measures to ensure human oversight and control of the AI system in compliance with regulatory requirements and organizational risk management.

Auditing Guidelines for Model Providers (MP)

1. Verify that the Model Provider (MP) has defined processes, procedures, and technical measures to ensure that AI systems are designed and developed in such a way that human operators can oversee their functioning and intended performance throughout their entire lifecycle. Ensure that the processes are documented in detail, covering scope, objectives, roles and responsibilities.

2. Examine the above-mentioned processes, procedures, and technical measures to confirm their compliance with relevant regulatory requirements and industry best practices.

3. Examine whether the above-mentioned processes, procedures, and technical measures adopt a risk-based approach.

4. Confirm that the above-mentioned processes, procedures, and technical measures are concretely and appropriately implemented by responsible parties over the entire AI systems’ lifecycle (from the design and market placement to the maintenance/upgrade and decommission phases).

5. Inspect whether the above-mentioned processes, procedures, and technical measures are monitored against sets of efficacy and efficiency metrics / indicators.

6. Inspect whether the above-mentioned processes, procedures, and technical measures are periodically reviewed and updated by responsible parties.

HRS: Human Resources

HRS-01: Background Screening Policy and Procedures

Control Specification

Establish, document, approve, communicate, apply, evaluate and maintain policies and procedures for background verification of all new employees (including but not limited to remote employees, contractors, and third parties) according to local laws, regulations, ethics, and contractual constraints and proportional to the data classification to be accessed, the business requirements, and acceptable risk. Review and update the policies and procedures at least annually, or upon significant changes.

Auditing Guidelines for Model Providers (MP)

1. Policy Documentation and Approval: Verify that the MP has a documented, approved background screening policy applying to employees, contractors, and third parties, proportional to role risk. Confirm the policy explicitly applies to data scientists, ML engineers, and infrastructure personnel with access to proprietary models or sensitive training data.

2. Defined Criteria: Verify that the policy defines consistent screening criteria: criminal history, employment history, education, professional licenses, and (if relevant) credit checks.

3. Transparency and Consent: Verify the policy is clearly communicated to applicants and written consent is obtained, following fairness and applicable laws.

4. Use of Providers: Verify the MP uses reputable, compliant background check providers.

5. Handling Adverse Findings: Verify the MP defines fair processes for addressing adverse findings, allowing candidates to respond or appeal.

6. Data Privacy and Security: Verify personal data collected through background checks is secured and handled in compliance with applicable privacy regulations.

7. Review and Update: Verify that the policy is reviewed and updated at least annually or after significant legal/regulatory changes.

8. Evidence of Compliance: Review a sample of hiring records to ensure background checks were completed before granting privileged access.

HRS-02: Acceptable Use of Technology Policy and Procedures

Control Specification

Establish, document, approve, communicate, apply, evaluate and maintain policies and procedures for defining allowances and conditions for the acceptable use of organizationally-owned or managed assets. Review and update the policies and procedures at least annually, or upon significant changes.

Auditing Guidelines for Model Providers (MP)

1. Policy Establishment and Documentation: Verify that the MP has established and formally documented an Acceptable Use Policy (AUP) that governs employee and contractor use of organizationally‑owned or managed assets, including Development and training environments, Model training pipelines and compute clusters, and Proprietary model weights, intellectual property (IP), and datasets.

2. Policy Communication and Acknowledgement: Confirm that the AUP is communicated to all internal personnel and contractors, and that acknowledgements are documented and retained.

3. Content of the AUP: Verify that the AUP explicitly includes: Prohibitions against Using unvetted, unauthorized, or non‑compliant training data sources, Exfiltrating, leaking, or misusing proprietary model weights or intellectual property, Ethical use of compute resources: Guidelines ensuring responsible, efficient, and ethical use of organizational compute infrastructure for model experimentation. General organizational expectations: Prohibitions on illegal, harassing, or unethical activities using organizational assets. Consequences for violations: Documented and communicated disciplinary or remedial actions.

4. Monitoring and Enforcement: Verify that monitoring controls are implemented to detect violations of AUP provisions, such as Logs and alerts for suspicious data uploads/downloads and Access controls and audit trails on model repositories. Ensure documented processes exist for investigating and enforcing consequences of violations.

5. Periodic Review and Maintenance: Confirm that the AUP is reviewed and updated at least annually, or after significant changes to regulatory requirements, technologies, or business priorities. Verify that review records and updated versions are retained.

6. Feedback and Continuous Improvement: Check whether feedback from employees, audit findings, or incidents is incorporated into periodic updates to improve the AUP over time.

HRS-03: Clean Desk Policy and Procedures

Control Specification

Establish, document, approve, communicate, apply, evaluate and maintain policies and procedures that require unattended workspaces to not have openly visible confidential data. Review and update the policies and procedures at least annually, or upon significant changes.

Auditing Guidelines for Model Providers (MP)

1. Confirm that Model Provider (MP) has a documented and implemented a policy requiring confidential model-related data (e.g., training data, model artifacts, prompts, outputs) not be visible in unattended workspaces.

2. Verify the policy is formally approved by relevant stakeholders (e.g., ML governance, security teams) and version-controlled.

3. Ensure the policy is communicated to all personnel involved in model development, deployment, and operations (e.g., ML engineers, data scientists, DevOps).

4. Check that enforcement mechanisms are in place, such as screen lock policies, clean desk protocols, and workspace monitoring.

5. Review incident logs for any breaches involving unattended exposure of sensitive model-related data.

6. Confirm the policy is reviewed and updated at least annually to reflect changes in model risk, regulatory requirements, or operational practices.

HRS-04: Remote and Home Working Policy and Procedures

Control Specification

Establish, document, approve, communicate, apply, evaluate and maintain policies and procedures to protect information accessed, processed or stored at remote sites and locations. Review and update the policies and procedures at least annually, or upon significant changes.

Auditing Guidelines for Model Providers (MP)

1. Verify the MP’s remote work policy includes specific safeguards for accessing and handling sensitive training data and model artifacts from remote locations.

2. Confirm that access to high-value research environments is restricted and heavily monitored for remote connections.

3. Verify if the Remote and Home Working Policy and associated procedures are reviewed and updated at least annually or upon significant changes in legal or regulatory requirements, information security or operational risk, business or workforce model, technology controls, or assurance and audit expectations.

HRS-05: Asset returns

Control Specification

Establish and document procedures for the return of organization-owned assets by terminated employees, contractors and third parties.

Auditing Guidelines for Model Providers (MP)

1. Verify the MP’s offboarding procedure ensures the return of all assets, especially high-powered workstations or devices containing proprietary research, training data, or model artifacts.

2. Confirm that access to code repositories and model registries is revoked immediately upon termination.

HRS-06: Employment Termination

Control Specification

Establish, document, and communicate to all relevant personnel the procedures outlining the roles and responsibilities concerning changes in employment.

Auditing Guidelines for Model Providers (MP)

1. Verify the MP’s termination procedures include immediate revocation of access to all model development environments, training datasets, model registries, and research repositories.

2. Confirm that a knowledge transfer process is in place to retain critical information about model development and research.

HRS-07: Employment Agreement Process

Control Specification

Employees sign the employee agreement prior to being granted access to organizational information systems, resources and assets.

Auditing Guidelines for Model Providers (MP)

1. Verify the MP’s onboarding process includes signing a robust employment agreement with strong IP and confidentiality clauses related to models and training data before any access to research or development environments is granted.

2. Review HR records for compliance.

HRS-08: Employment Agreement Content

Control Specification

The organization includes within the employment agreements provisions and/or terms for adherence to established information governance and security policies.

Auditing Guidelines for Model Providers (MP)

1. Review the MP’s employment agreement to ensure it contains strong provisions on intellectual property rights for AI models, confidentiality of training data, and adherence to ethical AI and responsible research policies.

2. Confirm the agreement is reviewed by legal counsel.

HRS-09: Personnel Roles and Responsibilities

Control Specification

Establish, document and communicate roles and responsibilities of employees, as they relate to information assets’ security and privacy.

Auditing Guidelines for Model Providers (MP)

1. Verify that job descriptions for the MP’s data scientists and ML engineers include responsibilities for training data security, model integrity, and adherence to responsible AI principles.

2. Review the MP’s model governance documentation for a clear RACI chart.

HRS-10: Non-Disclosure Agreements

Control Specification

Identify, document, and review, at planned intervals, requirements for non-disclosure/confidentiality agreements reflecting the organization’s needs for the protection of data and operational details.

Auditing Guidelines for Model Providers (MP)

1. Evaluate whether the AI Model Provider has clearly defined and documented its non-disclosure and confidentiality requirements, with specific focus on safeguarding proprietary AI models, training datasets, model weights, and prompt engineering techniques; ensuring confidentiality of customer data, usage patterns, and intellectual property; and managing third-party access, API integrations, and contractual confidentiality obligations.

2. Confirm that these non-disclosure and confidentiality agreements are periodically reviewed, ensuring they comply with internal policies and applicable regulations; adapt to changes in technology, threat landscape, or business operations; and undergo updates and re-approval as needed.

HRS-11: Security Awareness Training

Control Specification

Establish, document, approve, communicate, apply, evaluate and maintain a security awareness training program for all employees of the organization and provide regular training updates.

Auditing Guidelines for Model Providers (MP)

1. Verify the MP’s security awareness training includes specialized content on adversarial ML, data poisoning threats, and the protection of intellectual property related to AI models.

2. Check for training on the ethical implications of model development.

HRS-12: Personal and Sensitive Data Awareness and Training

Control Specification

Provide employees with access to sensitive organizational and personal data with appropriate security awareness training and regular updates in organizational procedures, processes, and policies relating to their professional function relative to the organization.

Auditing Guidelines for Model Providers (MP)

1. Confirm that personnel with access to sensitive model data (e.g., training datasets, model weights, prompts, outputs) receive security training. For example, ML engineers working on proprietary LLMs must complete secure data handling training before accessing model artifacts.

2. Check for documented training policies and access-role mappings, such as policies specifying that only senior data scientists can access production model weights after completing annual security refreshers.

3. Verify that training is completed and regularly updated to reflect evolving AI risks for instance may include topics like prompt injection, model inversion, and synthetic data leakage.

4. Ensure training is tailored to specific roles (e.g., ML engineers, data scientists, model evaluators). For example, DevOps staff receiving secure deployment training, while researchers focus on data anonymization and ethical model use.

5. Interview staff to confirm awareness of responsibilities and recent updates. For example, ask a model evaluator how they handle sensitive output logs and whether they’re aware of the latest clean desk policy.

6. Review how updates are communicated, such as through internal alerts, team briefings, or monthly AI governance bulletins that highlight changes in model risk protocols and link to updated training modules.

HRS-13: Compliance User Responsibility

Control Specification

Make employees aware of their roles and responsibilities for maintaining awareness and compliance with established policies and procedures and applicable legal, statutory, or regulatory compliance obligations.

Auditing Guidelines for Model Providers (MP)

1. Review how the Model Provider (MP) identifies and updates applicable AI-related legal, statutory, and regulatory obligations (e.g., ISO 42001, EU AI Act, U.S. state-level AI laws).

2. Collect evidence of documented processes, legal/compliance reviews, and involvement of relevant stakeholders (e.g., AI governance, legal, risk teams).

3. Interview staff (e.g., ML engineers, data scientists) to confirm awareness of their responsibilities under these obligations.

4. Check for role-specific training, signed acknowledgments, and ongoing compliance communications.

HRS-14: AI Competency Training

Control Specification

Establish, document, approve, communicate, apply, evaluate and maintain policies and procedures defining the AI training program for all relevant personnel of the organization based on their roles and provide regular training updates.

Auditing Guidelines for Model Providers (MP)

1. Verify the model provider (MP) has an approved AI training policy aligned with model development, deployment, and customer usage (e.g., covering responsible model training, fine-tuning, and release practices).

2. Examine that the training program defines role-specific paths (e.g., researchers on ethical model design, engineers on deployment safeguards, sales/support on responsible use cases).

3. Ensure training is accessible and delivered through onboarding, internal portals, or team-based sessions.

4. Review participation records to confirm staff receive training relevant to their roles in the model lifecycle.

5. Evaluate effectiveness through assessments or feedback, and confirm updates follow incidents, misuse, or audits.

6. Confirm training content is regularly updated to reflect new model capabilities, regulatory changes, or customer risk profiles.

HRS-15: AI Acceptable Use

Control Specification

Establish, document, and communicate to all personnel the policies and procedures on the acceptable use of AI technologies within the organization.

Auditing Guidelines for Model Providers (MP)

1. Confirm the Model Provider has a documented Acceptable Use Policy (AUP) for model development and deployment, approved by governance (e.g., prohibiting training for deepfakes or disinformation).

2. Ensure the AUP is accessible and clearly defines acceptable use (e.g., restricting models that enable impersonation or surveillance, requiring safeguards for high-risk applications).

3. Verify the policy is communicated through onboarding and role-specific training (e.g., research teams trained on ethical model design, sales/support trained to vet customer use cases).

4. Assess enforcement mechanisms like API monitoring and abuse detection (e.g., flagging prompt injection or policy-violating outputs, with consistent enforcement).

5. Check that the policy is regularly reviewed and updated (e.g., when releasing new model versions or enabling sensitive features like code generation or biometrics).

IAM: Identity & Access Management

IAM-01: Identity and Access Management Policy and Procedures

Control Specification

Establish, document, approve, communicate, implement, apply, evaluate and maintain policies and procedures for identity and access management. Review and update the policies and procedures at least annually, or upon significant changes.

Auditing Guidelines for Model Providers (MP)

1. Verify that a documented IAM policy exists specific to the development, training, and deployment environments of AI models.

2. Confirm that the IAM policy includes access restrictions for model weights, datasets, and fine-tuning tools.

3. Ensure there is documented evidence of periodic IAM policy reviews, especially after updates to AI architectures or tooling stacks.

4. Check that policy enforcement is automated using role-based access controls within development pipelines and CI/CD tools.

5. Confirm that policies include approval processes for researchers and developers accessing high-risk or sensitive model assets.

IAM-02: Credentials Management Policy and Procedures

Control Specification

Establish, document, approve, communicate, implement, apply, evaluate and maintain policies and procedures for the management of authentication credentials, including passwords. Review and update the policies and procedures at least annually, or upon significant changes.

Auditing Guidelines for Model Providers (MP)

1. Verify the Model Provider (MP) has established, documented, approved, and communicated policies governing authentication credentials for model training, development, deployment, and hosting environments, and confirm annual or event-driven review.

2. Confirm credential policies address both human users (e.g., data scientists, ML engineers) and non-human identities (e.g., training jobs, model pipelines, automation agents).

3. Verify MFA enforcement for privileged access to training datasets, model repositories, and deployment infrastructure.

4. Confirm credential lifecycle management processes include secure issuance, rotation, revocation, and protection of API keys, service accounts, and access tokens used in ML workflows.

5. Verify secure storage of credentials within model development and hosting environments, including encryption and access restrictions.

6. Confirm monitoring, logging, and periodic evaluation of authentication activities to detect misuse or unauthorized credential access.

IAM-03: Identity Inventory

Control Specification

Manage, store, and regularly review the inventory of identities, and monitor their level of access.

Auditing Guidelines for Model Providers (MP)

1. Confirm that all identities interacting with model training, validation, and deployment systems are inventoried.

2. Validate role-to-identity mapping for ML tools (e.g., Jupyter, Kubeflow, SageMaker).

3. Ensure inventories distinguish between human users, service accounts, and AI agents.

4. Verify that access to sensitive datasets or models is tied to valid identities in the inventory.

5. Assess that automated pipelines do not generate unmanaged or temporary credentials.

6. Check that inventory reviews occur after major updates to model infrastructure or personnel shifts.

IAM-04: Separation of Duties

Control Specification

Employ the separation of duties principle when implementing information system access.

Auditing Guidelines for Model Providers (MP)

1. Verify MP enforces SoD between model development, training, deployment, and monitoring.

2. Confirm that developers and validators have separate access rights.

3. Assess controls in place to prevent unauthorized changes by those validating model outputs.

4. Review policy defining SoD requirements specific to AI model lifecycle.

5. Check if SoD compliance is tracked and reviewed periodically.

IAM-05: Least Privilege

Control Specification

Employ the least privilege principle when implementing information system access.

Auditing Guidelines for Model Providers (MP)

1.Verify that documented policies are in place specifying adherence to the principle of least privilege, including defined responsibilities for role and access governance.

1. Confirm that privileges are assigned based strictly on job responsibilities, and that these assignments are periodically reviewed and adjusted to eliminate excessive or outdated permissions.

2. Verify that privileged activities are logged and monitored, and that any anomalies or suspicious activity are escalated according to documented procedures.

3. Ensure procedures exist to investigate and remediate violations of least privilege policies, including timelines, responsible roles, and incident resolution steps.

IAM-06: Access Provisioning

Control Specification

Define and implement an identity access provisioning process which authorizes, records, and communicates access changes to data and assets.

Auditing Guidelines for Model Providers (MP)

1. Ensure provisioning of access to ML models and datasets is approval-based.

2. Verify AI research and DevOps teams have clearly segregated access scopes.

3. Confirm logs track access grants to model repositories or training pipelines.

4. Check provisioning systems prevent unintended privilege propagation.

IAM-07: Access Changes and Revocation

Control Specification

De-provision or modify identity access in a timely manner.

Auditing Guidelines for Model Providers (MP)

1. Ensure access to training and inference environments is revoked upon role change or offboarding.

2. Verify process to immediately revoke access to sensitive AI model assets.

3. Confirm AI service accounts are disabled or rotated when no longer needed.

4. Review exceptions and delays in revocation of model contributors.

IAM-08: Access Review

Control Specification

Review and revalidate identity access for least privilege and separation of duties with a frequency that is commensurated with organizational risk tolerance and at least annually, or upon significant changes.

Auditing Guidelines for Model Providers (MP)

1. Check that access to training datasets, model artifacts, and inference pipelines are reviewed periodically.

2. Ensure identity-based audit logs are reviewed to confirm authorized model access.

3. Validate that outdated or stale access to model deployment infrastructure is revoked.

4. Confirm that research staff and operational users are segmented in access reviews.

5. Review change logs to verify adjustments in access rights post-review.

IAM-09: Segregation of Privileged Access Roles

Control Specification

Define, implement and evaluate processes, procedures and technical measures for the segregation of privileged access roles.

Auditing Guidelines for Model Providers (MP)

1. Confirm that model training, tuning, and deployment roles are segmented by function.

2. Verify role separation between data engineers and MLOps.

3. Check privilege logs for evidence of role misuse or bypassing controls.

4. Ensure that no individual holds access to both raw training data and production models.

5. Review access provisioning history for cases of inappropriate dual-role assignments.

6. Confirm segregation policies are explicitly documented and enforced.

IAM-10: Management of Privileged Access Roles

Control Specification

Define and implement an access process to ensure privileged access roles and rights are granted for a time limited period, and implement procedures to prevent the accumulation of segregated privileged access.

Auditing Guidelines for Model Providers (MP)

1. Confirm that ML pipeline administrators and data owners hold distinct privileged roles.

2. Validate review processes for privileged access to model registries and training environments.

3. Check if identity-based monitoring exists for privileged ML operations.

4. Verify use of short-term access tokens for critical model operations.

5. Review privilege distribution among model validators, researchers, and deployers.

IAM-11: Service Customers’ Approval for Agreed Privileged Access Roles

Control Specification

Define, implement and evaluate processes and procedures for service customers to participate, where applicable, in the granting of access for agreed, high risk (as defined by the organizational risk assessment) privileged access roles.

Auditing Guidelines for Model Providers (MP)

1. Verify privileged roles related to model training, deployment, or retraining are documented and segregated to prevent overlap between development, approval, and deployment functions.

2. Check that a formal approval process exists for model-access roles, especially those impacting inferencing.

3. Confirm customer-defined access controls or approval involvement for AI model manipulation roles.

4. Review records of role assignments and revocations.

5. Validate that approvals are logged and independently reviewed.

6. Confirm periodic assessments of role effectiveness and business relevance.

7. Ensure segregation of roles to prevent unreviewed escalation paths.

IAM-12: Unique Identities

Control Specification

Define, implement and evaluate processes, procedures and technical measures, that ensure identities’ activities are identifiable through uniquely associated IDs.

Auditing Guidelines for Model Providers (MP)

1. Verify all individuals with access to model training or deployment environments are uniquely identified.

2. Confirm service accounts used in automated model pipelines are documented and monitored.

3. Validate linkage between user identity and model versioning/auditing processes.

4. Check for authentication records tying actions back to individuals during model modifications.

5. Confirm governance exists over account provisioning and deactivation.

IAM-13: Strong Authentication

Control Specification

Define, implement and evaluate processes, procedures and technical measures for authenticating access to systems, application and data assets, including multifactor authentication for at least privileged user and sensitive data access. Adopt digital certificates or alternatives which achieve an equivalent level of security for system identities.

Auditing Guidelines for Model Providers (MP)

1. Confirm that documented policies mandate strong authentication mechanisms, including password complexity, multi-factor authentication (MFA), and device-level security measures.

2. Verify that MFA is implemented for access to privileged accounts, administrative interfaces, and systems handling sensitive or regulated data.

3. Ensure that system identities (e.g., service accounts, APIs, machine workloads) are authenticated using digital certificates, cryptographic keys, or other strong credentials.

4. Check that authentication methods comply with current best practices and industry standards (e.g., NIST SP 800-63, OWASP ASVS).

5. Confirm that technical controls are in place to enforce authentication requirements, such as conditional access, session timeouts, and login attempt limits.

6. Ensure regular audits are conducted to validate the effectiveness and adherence of authentication mechanisms.

7. Verify that documented procedures exist for credential recovery, reset, and revocation, including secure identity verification steps.

8. Review monitoring mechanisms that detect authentication anomalies, such as brute-force attempts, logins from unusual geolocations, or session hijacking indicators.

IAM-14: Credentials Management

Control Specification

Define, implement and evaluate processes, procedures and technical measures for the secure management of authentication credentials, including passwords.

Auditing Guidelines for Model Providers (MP)

1. Verify credentials used by model training/inference systems are securely stored and scoped.

2. Check for secret rotation policies for model-serving and data access tokens.

3. Ensure credentials for accessing model repositories are encrypted and access-controlled.

4. Confirm controls are in place to prevent leakage of credentials in logs or crash dumps.

5. Validate that training pipelines use temporary credentials for sensitive data access.

IAM-15: Authorization Mechanisms

Control Specification

Define, implement and evaluate processes, procedures and technical measures to verify access to data and system functions is authorized.

Auditing Guidelines for Model Providers (MP)

1. Verify authorization is required for all model training, updating, or output modification actions.

2. Check that only approved identities can access sensitive training data or parameter configurations.

3. Confirm access control lists (ACLs) are applied at the model repository level.

4. Validate logging of all authorization decisions tied to AI model usage.

5. Ensure periodic validation of role-to-permission mappings.

IAM-16: Knowledge Access Control - Need to Know

Control Specification

Define policy and procedure for “need to know” access to knowledge, information and data within the organization and in the context of the AI system to be applied when regulating access to resources.

Auditing Guidelines for Model Providers (MP)

1. Confirm policies explicitly define access limits for model design, hyperparameters, and architecture blueprints.

2. Verify that model developers and data scientists are granted access only to components necessary for their roles.

3. Check access control logs for model tuning and retraining datasets.

4. Ensure version control repositories (e.g., for model weights) enforce restricted contributor permissions.

5. Validate procedures for securely sharing model internals with external researchers or partners.

6. Verify role-based access for experimentation on proprietary training datasets and augmentation logic.

IAM-17: Output Modification and Special Authorization

Control Specification

When allowing model output modification of AI generated output, establish a role for this access and allow changes only by authorized identities.

Auditing Guidelines for Model Providers (MP)

1. Ensure documented justifications are required for model output overrides (e.g., for regulatory alignment).

2. Verify only senior ML engineers or governance approvers can authorize output modification.

3. Confirm cryptographic hash or signature verification to ensure output integrity.

4. Validate regular reviews of model post-processing scripts and filters.

IAM-18: Agent Access Restriction

Control Specification

Restrict agents’ access to the tools and plugins necessary for the activity or use case at hand, ensuring adherence to the principles of need-to-know and least privilege.

Auditing Guidelines for Model Providers (MP)

1. Confirm user identities used during model training or fine-tuning are mapped securely and revocably.

2. Validate that user consent is collected before training on identifiable or sensitive inputs.

3. Ensure controls prevent model memorization or exposure of revoked identities during inference.

4. Verify retraining workflows accommodate removal of revoked user records.

IPY: Interoperability & Portability

IPY-01: Interoperability and Portability Policy and Procedures

Control Specification

Establish, document, approve, communicate, apply, evaluate and maintain policies and procedures for interoperability and portability including requirements for: a. Communications between application interfaces b. Information processing interoperability c. Application development portability d. Information/Data exchange, usage, portability, integrity, and persistence Review and update the policies and procedures at least annually or upon significant changes.

Auditing Guidelines for Model Providers (MP)

1. Verify the existence of documented policies and procedures addressing interoperability and portability ensuring it contains communications between application interfaces, information processing interoperability, application development portability.

2. Confirm that the policies and procedures have received appropriate approval from relevant authority within the MP’s organization.

3. Examine evidence of a regular review and update cycle, ensuring the policies and procedures are evaluated and updated.

4. Verify the Application Provider’s due diligence process for ensuring that upstream providers implement controls related to interoperability and portability.

5. Verify that the policies are effectively communicated to relevant stakeholders, including internal personnel and any external partners or service customers impacted by these controls.

6. Verify that the review and update of the interoperability and portability policies and procedures occur at least annually, and that evidence of review (e.g., change logs, approvals) is retained.

IPY-02: Application Interface Availability

Control Specification

Provide application interface(s) to service customers so that they programmatically retrieve their data to enable interoperability and portability.

Auditing Guidelines for Model Providers (MP)

1. Verify that the Model Provider has established clear policies governing application interface availability, and that functional APIs are implemented to allow AI service customers to retrieve data from models (e.g., predictions, outputs) under defined conditions.

2. Verify that the policies clearly define the categories of users (e.g., service customers, internal personnel) who are subject to guidelines with respect to scope and applicability of application interfaces (APIs).

3. Evaluate that the policies explicitly specify what constitutes acceptable and prohibited use of the application interfaces.

4. Ensure that these policies are up-to-date, reflecting the latest technological standards and regulatory requirements.

IPY-03: Secure Interoperability and Portability Management

Control Specification

Implement cryptographically secure network protocols for the management, import and export of data, according to industry standards.

Auditing Guidelines for Model Providers (MP)

1. Examine Interoperability and Portability Policy for adequacy, currency, and communication.

2. Verify inclusion of applicable data management scenarios

3. Specify Secure Protocols and Cryptographic Standards: Ensure there is a formal acknowledgment process for policy acceptance, such as training sessions, signed acknowledgments, or digital records of policy acceptance.

IPY-04: Data Portability Contractual Obligations

Control Specification

Agreements must include provisions specifying service customers’ access to data upon contract termination and will include: a. Data format b. Length of time the data will be stored c. Scope of the data retained and made available to the service customers d. Data deletion policy

Auditing Guidelines for Model Providers (MP)

1. Review existing contracts and agreements between the Model Provider and AICs to verify the inclusion of data portability provisions, including model weights, logs, and metadata.

2. Verify that the contracts explicitly state the formats in which data is available to facilitate easy data migration and interoperability.

3. Review contract agreements to ensure which types of data might be available and which might be deleted post termination.

4. Review the data deletion process to verify the data deletion methods, processes and timelines by which data will be securely deleted as specified in the contract agreement.

I&S: Infrastructure Security

I&S-01: Infrastructure and Virtualization Security Policy and Procedures

Control Specification

Establish, document, approve, communicate, apply, evaluate and maintain policies and procedures for infrastructure and virtualization security. Review and update the policies and procedures at least annually, or upon significant changes.

Auditing Guidelines for Model Providers (MP)

1. Examine Model Provider (MP) policies and procedures that defines the scope, objectives, roles, and responsibilities for securing Model infrastructure and virtualization environments.

2. Verify the Policies and Procedures ensure third-party (e.g., OSP, AP, CSP) reviews and controls are place and considered in the MP procedures.

3. Verify that policies and procedures are documented and approved from senior management or governing authority and update versioning.

4. Determine that approved policies and procedures are communicated to all relevant internal and external parties, ensuring they read, understand, and fulfill their roles and responsibilities.

5. Verify policies and procedure are effectively applied in daily operations and evaluated continuously for operational effectiveness and compliance.

6. Verify that security policies and procedures are regularly reviewed and updated to address emerging threats, vulnerabilities, and evolving business needs, ensuring clear documentation of changes and approvals exists.

I&S-02: Capacity and Resource Planning

Control Specification

Plan and monitor the availability, quality, and adequate capacity of resources in order to deliver the required system performance as determined by the business.

Auditing Guidelines for Model Providers (MP)

1. Examine the Model Provider’s capacity and resource planning document that explicitly addresses AI resource availability, quality, and capacity to meet business objectives and service level agreements. This should include scope, roles, responsibilities, and processes for forecasting, scaling and monitoring resources.

2. Verify capacity plans, performance forecasts, and scaling procedures are reviewed and approved by senior management or governance authorities.

3. Verify model training and deployment pipelines are optimized for resource efficiency, identifying any potential underutilization or over-provisioning of AI resources.

4. Determine scaling mechanisms and failover strategies are in place to handle AI workloads.

5. Verify performance planning procedures regularly review, at least annually and align with changing business demands, system performance metrics, emerging technologies, and evolving threats.

I&S-03: Network Security

Control Specification

Monitor, encrypt and restrict communications between environments, services, and applications to only authenticated and authorized connections, as justified by the business. Review these configurations at least annually, and support them by a documented justification of all allowed services, protocols, ports, and compensating controls.

Auditing Guidelines for Model Providers (MP)

1. Examine AI/ML model–related network security policies and procedures to ensure they define approved communication paths, access restrictions, and encryption requirements consistent with organizational and regulatory security requirements.

2. Determine network access controls, encryption, and integrity mechanisms for model security.

3. Verify that network controls restrict access to model deployment and update endpoints such that only authenticated and authorized entities can deploy or modify models.

4. Verify continuous network monitoring, logging, and anomaly detection for AI/ML model–related network traffic and communications.

5. Verify regular reviews, at least annually, with structured record-keeping of changes and approvals.

I&S-04: OS Hardening and Base Controls

Control Specification

Harden host and guest OS, hypervisor or infrastructure control plane, according to their respective best practices, and supported by technical controls, as part of a security baseline.

Auditing Guidelines for Model Providers (MP)

1. Examine documented security policies and hardening baselines to ensure alignment with industry best practices and AI model hosting requirements.

2. Determine if system hardening measures, such as secure configurations for host OS, virtual machines, and cloud instances, are enforced for AI model environments.

3. Verify an annual review of hardening configurations, including host OS, hypervisors, and AI model deployment environments, ensuring results are reviewed and approved.

4. Verify regular security assessments against established security baselines, ensuring prompt remediation of identified vulnerabilities.

5. Verify if emerging threats and evolving security risks are monitored, ensuring OS hardening procedures are updated with structured record-keeping of changes and approvals.

I&S-05: Production and Non-Production Environments

Control Specification

Separate production and non-production environments to reduce the risk of sensitive production data being used in non-production environments. Production data is sanitized or protected before any authorized non-production use.

Auditing Guidelines for Model Providers (MP)

1. Examine Model Provider (MP) policies and procedures ensuring separation between development, training, testing, and production environments to reduce the risk of sensitive production data exposure.

2. Verify role-based and least-privilege access controls enforce separation between production and non-production model environments, restricting production access to authorized personnel only.

3. Verify production data used for testing, validation, retraining, or experimentation in non-production environments is sanitized or otherwise protected before any authorized use.

4. Verify formal model promotion and deployment processes between non-production and production environments preserve environment separation and prevent unauthorized data transfer, including documented approvals and integrity validation.

5. Verify monitoring and logging across all model environments to detect unauthorized access, data movement, or model changes, and confirm logs are securely maintained and reviewed.

6. Confirm segregation and production data protection controls are periodically reviewed and updated to address evolving AI security risks and compliance requirements.

I&S-06: Segmentation and Segregation

Control Specification

Design, develop, deploy and configure applications and infrastructures such that service customer (tenant) access is appropriately segmented and segregated, monitored and restricted.

Auditing Guidelines for Model Providers (MP)

1. Verify documented policies and procedures define how service customer (tenant) isolation is enforced within model serving, customer-specific model artifacts, and associated data handling processes operated by the Model Provider.

2. Verify logical segmentation mechanisms are implemented to isolate service customer (tenant) model artifacts, customer-provided or customer-specific datasets, inference contexts, and supporting systems, preventing unauthorized cross-tenant access or data leakage.

3. Verify access controls governing model access, customer-specific training or customization workflows (where applicable), inference endpoints, and model management functions restrict identities and permissions to their assigned service customer (tenant) context, consistent with segmentation policies.

4. Evaluate whether segmentation and isolation controls are periodically tested or validated to confirm that service customer (tenant) boundaries cannot be bypassed through model access paths, customer-specific customization workflows (if applicable), inference requests, or artifact reuse.

5. Confirm incident response procedures address failures of service customer (tenant) isolation related to model access, customer data exposure, inference misuse, or unauthorized model modification.

6. Verify logging and monitoring mechanisms are configured to detect and alert on unauthorized cross-tenant activity involving model access, inference usage, customer-specific customization operations (if applicable), or model artifact handling.

7. Confirm relevant personnel receive training on model-specific segmentation responsibilities, including risks associated with configuration errors or operational practices that could compromise service customer (tenant) isolation.

I&S-07: Migration to Hosted Environments

Control Specification

Use secure and encrypted communication channels when migrating servers, services, applications, or data to hosted environments. Such channels must include only up-to-date and approved protocols.

Auditing Guidelines for Model Providers (MP)

1. Verify the Model Provider (MP) documented migration procedures explicitly require secure, encrypted communication channels.

2. Confirm encryption mechanisms adhere to current security standards (e.g., ensuring data privacy, model security, and communications with users).

3. Check records documenting secure migration processes.

4. Ensure risk assessments conducted before migrating sensitive data to cloud environments.

5. Validate compliance checks post-migration to confirm the security and integrity of data.

6. Confirm clearly defined roles and responsibilities for migration activities.

7. Verify documented incident response plans for issues arising during cloud migration.

I&S-08: Network Architecture Documentation

Control Specification

Identify and document high-risk environments based on data sensitivity, threat exposure, and business impact.

Auditing Guidelines for Model Providers (MP)

1. Verify the Model Provider has comprehensive documentation identifying and detailing high-risk environments based on data sensitivity, threat exposure, and business impact.

2. Confirm regular updates and reviews of network architecture documentation (e.g., Threat Models, Reviews with OWASP top 10 LLM, MITRE Atlas, and risk assessments, AI Red Teaming).

3. Check availability and accessibility of architecture documentation to authorized personnel.

4. Ensure documentation aligns with current network configurations and practices.

5. Validate documented processes for identifying and managing changes to network architecture.

6. Confirm training provided to responsible personnel for maintaining accurate documentation.

I&S-09: Network Defense

Control Specification

Define, implement and evaluate processes, procedures and defense-in-depth techniques for protection, detection, and timely response to network-based attacks.

Auditing Guidelines for Model Providers (MP)

1. Verify the Model Provider (MP) documented procedures clearly define network defense mechanisms.

2. Confirm regular implementation and evaluation of defense strategies (e.g., signing compiled solutions, fingerprinting models, IDS/IPS).

3. Check routine testing of defense mechanisms for effectiveness against current threats.

4. Ensure monitoring and logging effectively capture events relevant to network defense.

5. Validate timely response and mitigation processes for detected threats.

6. Confirm clear accountability and documented roles for network defense management.

7. Verify regular training sessions on network defense practices provided to security teams.

LOG: Logging and Monitoring

LOG-01: Logging and Monitoring Policy and Procedures

Control Specification

Establish, document, approve, communicate, apply, evaluate and maintain policies and procedures for logging and monitoring. Review and update the policies and procedures at least annually, or upon significant changes.

Auditing Guidelines for Model Providers (MP)

1. Conduct interviews with personnel responsible for documenting, maintaining, and communicating organizational logging and monitoring policies, procedures, and standards (the Policies).

2. Inspecting Records and Documents: Obtain and review the Policies to ensure they are adequate for the organization to manage risks associated with logging and monitoring. Verify that the Policies define the personnel or roles responsible for their dissemination, identify an official accountable for managing the Policies, specify the frequency of reviews and updates (annually), and outline events that necessitate policy updates. Review the Policies by performing the following verification steps:

   1. Ensure policies include logging of model lifecycle activities such as training, validation, and deployment.

   2. Verify that access to logs related to model behavior and model drift detection is restricted and auditable.

   3. Confirm that logs capture interventions made to models in production, including human-in-the-loop overrides.

   4. Validate log retention periods for AI-specific logs in accordance with internal policies and regulatory guidelines.

   5. Check procedures to monitor model performance degradation or unusual inference patterns in logs.

   6. Ensure audit logs reflect the use of synthetic or sensitive training data where applicable.

   7. Confirm centralized log visibility for monitoring AI behavior across environments.

3. Verify that the Policies are communicated, reviewed and updated at least annually or upon significant changes, are approved, and communicated to relevant stakeholders.

LOG-02: Audit Logs Protection

Control Specification

Define, implement and evaluate processes, procedures and technical measures to ensure the security and retention of audit logs.

Auditing Guidelines for Model Providers (MP)

1. Inquiring with Control Owners

   1. Conduct interviews with personnel responsible for defining, implementing, and evaluating audit log security and retention processes for AI model development, training, and distribution to understand their roles in protecting model lifecycle logs and research data. Verify their understanding of technical measures implemented to ensure audit logs from model training, fine-tuning, evaluation, and distribution activities remain secure and are retained according to organizational requirements.

2. Inspecting Records and Documents

   1. Verify that model training logs, evaluation metrics, and inference logs are stored in write-once or append-only formats where feasible.

   2. Confirm that logs containing proprietary model information, training data references, and performance metrics are protected using encryption at rest and in transit.

   3. Ensure access to AI model logs is restricted to authorized research and development personnel only, with RBAC or IAM controls protecting intellectual property.

   4. Validate mechanisms to detect and alert on unauthorized access attempts or changes to logs containing model parameters, training data, or research findings.

   5. Check that AI model log protection is periodically tested through internal audits, including validation of intellectual property protection controls.

   6. Confirm that log retention for model development data aligns with research requirements, intellectual property policies, and regulatory compliance requirements.

   7. Verify that controls are in place to prevent unauthorized modification of model training logs and version control records.

   8. Review documented processes and procedures for AI model audit log security, including data provenance and model governance procedures.

   9. Validate that backup and recovery procedures exist for critical AI model logs to ensure research continuity and model reproducibility.

   10. Confirm that log disposal procedures are secure and documented for model development logs when retention periods expire, including protection of trade secrets.

   11. Verify the use of cryptographic integrity checks or write-once mechanisms for critical AI logs.

   12. Confirm procedures to detect unauthorized changes to logs involving model decisions or outputs.

   13. Check that logging access is segregated by function (e.g., engineering vs. data science).

   14. Validate that protection measures extend to logs containing PII or training data lineage.

   15. Confirm logs are retained in accordance with model lifecycle management policies.

   16. Ensure forensic readiness by preserving protected logs for incident analysis.

LOG-03: Security Monitoring and Alerting

Control Specification

Identify and monitor security-related events within applications, the underlying infrastructure. Define and implement a system to generate alerts to responsible stakeholders based on such events and corresponding metrics.

Auditing Guidelines for Model Providers (MP)

1. Inquiring with Control Owners: Conduct interviews with personnel responsible for identifying, monitoring, and alerting on security-related events within model development environments, training infrastructure, model storage systems, distribution platforms, and inference environments. Determine how security events are defined and classified, how monitoring thresholds or metrics are established, and how alert notifications are routed to responsible stakeholders for model-related security incidents (e.g., unauthorized access, model exfiltration, training data compromise, or model integrity tampering).

2. Inspecting Records and Documents

   1. Verify documented procedures define security-related events to be monitored across model development, training, storage, distribution, and inference environments.

   2. Verify monitoring mechanisms are implemented to detect defined security events across model development, training, storage, distribution, and inference environments, including security-relevant interactions between infrastructure components (e.g., unauthorized model access, model exfiltration, training data access violations, model parameter extraction attempts, unauthorized modification or fine-tuning, and data poisoning indicators).

   3. Review logs or monitoring outputs to determine whether defined security events are captured and retained in accordance with monitoring procedures.

   4. Verify alerting mechanisms are configured and generate notifications when defined security event thresholds or metrics are met.

   5. Verify that alert notifications are directed to appropriate responsible stakeholders and that roles for triage and escalation of model-related security incidents are defined.

   6. Review evidence that monitoring configurations and alert thresholds are periodically reassessed and updated as model architecture, infrastructure, or threat conditions change.

LOG-04: Audit Logs Access and Accountability

Control Specification

Restrict audit log access to authorized identities and maintain records of that access.

Auditing Guidelines for Model Providers (MP)

1. Inquiring with Control Owners

   1. Conduct interviews with personnel responsible for managing AI model audit log access controls and maintaining access records to understand their authorization processes for accessing model training logs, inference logs, and model security events. Verify their understanding of access restriction mechanisms and record-keeping requirements for all AI model audit log access activities.

2. Inspecting Records and Documents

   1. Verify access to AI model-generated audit logs (including training events, inference requests, model performance metrics, and security events) is restricted to authorized personnel.

   2. Ensure AI model logging access is role-based and mapped to least privilege principles, protecting proprietary model information and research data.

   3. Confirm all AI model log access events are themselves logged with timestamps, actor IDs, and specific model data accessed.

   4. Check for formal review processes of AI model log access permissions, including intellectual property protection considerations.

   5. Validate AI model researchers and engineers are not granted persistent access to sensitive model logs without approval and research justification.

   6. Review incident records for unauthorized access to AI model audit logs and follow-up actions taken.

   7. Confirm procedures are in place to revoke AI model log access upon role changes or terminations.

   8. Examine documented access control policies and procedures for AI model audit log systems, including trade secret protections.

   9. Validate that AI model access records are retained according to intellectual property policies and regulatory requirements.

   10. Review monitoring and alerting mechanisms for unauthorized or suspicious AI model audit log access attempts.

   11. Confirm only authorized personnel (e.g., ML engineers) can access logs from training, inference, or model serving.

   12. Validate all access to sensitive logs (e.g., model queries, inference results) is logged and auditable.

   13. Ensure log access controls are applied consistently across experimentation and production environments.

   14. Verify logs containing sensitive training data or model behavior insights are secured with strict accountability.

   15. Review audit trails showing who accessed logs and when, especially for logs involving model output anomalies.

   16. Check mechanisms exist to detect and alert on unauthorized log access.

   17. Review segregation of duties policies to prevent unauthorized modification, deletion, or tampering of audit logs.

LOG-05: Audit Logs Monitoring and Response

Control Specification

Implement and maintain capabilities to correlate and monitor security audit logs for the detection of suspicious or anomalous activity that deviates from typical or expected patterns. Establish and follow a defined process to review and take appropriate and timely actions on detected anomalies.

Auditing Guidelines for Model Providers (MP)

1. Inquiring with Control Owners: Conduct interviews with personnel responsible for monitoring AI model security audit logs to understand how anomalous activity in model training, inference, and distribution environments is identified, how baseline model behavior is defined, and how detected anomalies are reviewed and acted upon in a timely manner.

2. Inspecting Records and Documents

   1. Verify security audit logs are generated and monitored for model lifecycle components, including training environments, inference endpoints, model repositories, and access activities.

   2. Verify correlation rules or detection mechanisms are implemented to identify suspicious or anomalous activity that deviates from defined baseline or expected model behavior.

   3. Review documentation defining baseline model operational patterns and criteria used to classify activity as anomalous.

   4. Review monitoring outputs or dashboards to determine whether detected anomalies are logged and tracked.

   5. Verify a documented process exists for reviewing detected model anomalies and taking appropriate and timely action.

   6. Inspect evidence that detected anomalies are reviewed and that appropriate and timely actions are taken and documented in accordance with the defined process.

   7. Review evidence that anomaly detection thresholds or correlation rules are periodically reassessed and updated as model architecture, training processes, or threat conditions change.

LOG-06: Clock Synchronization

Control Specification

Use a reliable time source across all relevant information processing systems.

Auditing Guidelines for Model Providers (MP)

1. Inquiring with Control Owners

   1. Conduct interviews with personnel responsible for managing time synchronization across AI model development and distribution systems to understand their implementation of reliable time sources for model training logs, research activities, and model versioning. Verify their understanding of time synchronization requirements for training infrastructure, model distribution platforms, and procedures for maintaining accurate timestamps across model lifecycle activities.

2. Inspecting Records and Documents

   1. Confirm AI model development systems handling training processes and model distribution use a centralized time source.

   2. Verify implementation of Network Time Protocol (NTP) or equivalent time synchronization protocols across model training clusters and research infrastructure.

   3. Check synchronization logs to validate accurate timestamping across model training processes, evaluation metrics, and distribution activities.

   4. Assess whether unsynchronized model development systems trigger alerts or errors that could affect training reproducibility or research integrity.

   5. Verify clock drift thresholds are defined and monitored for model training infrastructure and model serving systems.

   6. Confirm the accuracy of timestamps in model development logs critical for research reproducibility and intellectual property protection.

   7. Validate incident response records for model-related issues reference consistent timestamps across training sessions and model deployment events.

   8. Examine documentation of reliable time source configuration for model development environments and backup time synchronization mechanisms.

   9. Review time synchronization policies covering model training systems, research platforms, and model distribution infrastructure.

   10. Validate that time source reliability is monitored for model development activities and backup time sources are available to ensure research continuity.

   11. Ensure all model-serving environments and pipelines synchronize with a reliable time source.

   12. Verify timestamp alignment across model logs, inference requests, and security logs.

   13. Check whether clock sync mechanisms are included in deployment templates.

   14. Review system logs for anomalies due to clock mismatches during model training or serving.

   15. Confirm configuration compliance for time synchronization policies.

   16. Assess whether timing discrepancies impact forensic reconstruction

LOG-07: Logging Scope

Control Specification

Establish, document and implement which information meta/data system events should be logged. Review and update the scope at least annually or whenever there is a change in the threat environment, and as per relevant regulatory requirements.

Auditing Guidelines for Model Providers (MP)

1. Inquiring with Control Owners

   1. Conduct interviews with personnel responsible for establishing, documenting, and implementing logging scope for AI model development events and research activity metadata to understand their process for defining which model lifecycle events should be logged and their procedures for annual reviews. Verify their understanding of model security threat environment changes that trigger scope updates and their implementation of documented logging requirements across model training, evaluation, and distribution systems.

2. Inspecting Records and Documents

   1. Confirm documentation specifies which AI model events must be logged (e.g., training access, model evaluation, inference requests, model parameter changes, research data access).

   2. Validate inclusion of both success and failure events in the AI model logging scope, including successful training runs and failed model access attempts.

   3. Ensure regular reviews of AI model logging scope to capture evolving model security threats such as model stealing, training data poisoning, and unauthorized fine-tuning.

   4. Check scope alignment with intellectual property protection requirements, research integrity standards, and model distribution contractual obligations.

   5. Assess procedures for adjusting logging scope when deploying new model architectures, training methodologies, or distribution channels.

   6. Confirm stakeholder approval for the defined AI model logging scope, including input from research leadership, legal teams, and model safety officers.

   7. Verify logs reflect real-world AI model events as specified in scope documents, including training processes, evaluation metrics, and model access patterns.

   8. Examine evidence of annual AI model logging scope reviews and documentation of any scope updates driven by new model security threats or research policy changes.

   9. Review procedures for monitoring and responding to model threat environment changes that may require logging scope adjustments for emerging attack techniques.

   10. Validate that implementation of AI model logging scope requirements is consistently applied across all model development environments, training infrastructure, and distribution platforms.

   11. Validate that model lifecycle events (training, versioning, access) are in the logging scope.

   12. Confirm that input/output trace logs are generated where applicable.

   13. Verify scope includes abnormal model behavior or inference errors.

   14. Ensure that logs record access to training datasets or model weights.

   15. Confirm logging is enabled for model deployment and rollback events.

   16. Review defined scope against real model-serving scenarios and data flows.

   17. Confirm that changes to the defined logging scope are documented with version history, formal approvals, and traceability to relevant regulatory requirements or compliance obligations.

LOG-08: Audit Logs Sanitization

Control Specification

Define, implement and evaluate technical measures for service customers to detect and scrub or tokenize sensitive data from logs to prevent unauthorized exposure, as per applicable laws and regulations.

Auditing Guidelines for Model Providers (MP)

1. Inquiring with Control Owners: Conduct interviews with personnel responsible for defining, implementing, and evaluating the technical measures that allow service customers to detect, and scrub, or tokenize sensitive data from applicable, and scoped, AI logs, which helps customers prevent unauthorized exposure, as per applicable laws and regulations. Verify their understanding of this customer responsibility, and its applicability for the scoped product or service within the workflow the Model Provider supports.

2. Review product or service baseline, or agreement to verify that customer can opt-in for this responsibility, within the workflow the Model Provider supports, and based on the regulations that are applicable.

3. Verify that automated safeguards are in place according to the technical measures defined for the product or service within the workflow the Model Provider supports, and based on the regulations that are applicable.

   1. Review the product or service description.

   2. Review the product or service customer agreement.

   3. Review the product or service customer baseline.

4. For those customers that opt-in, examine logs of the scoped product or services to verify only allowed information exists within logs. Due to the fact that company policy may not allow a review of this nature, especially logs that are customer or partially-customer controlled, policy should be cited, and this step skipped. Additionally, it is possible to review the logs of a test environment if one is available.

LOG-09: Log Records

Control Specification

Generate audit records containing relevant security information.

Auditing Guidelines for Model Providers (MP)

1. Inquiring with Control Owners

   1. Conduct interviews with personnel responsible for generating audit records containing relevant AI model security information to understand their processes for capturing, formatting, and maintaining security-related audit data across model development, training, and distribution activities. Verify their understanding of what constitutes relevant model security information and their procedures for ensuring audit records contain sufficient detail for model security investigations, intellectual property protection, and research integrity requirements.

2. Inspecting Records and Documents

   1. Verify AI model logs capture event type, timestamp, actor, and source for all model training, evaluation, and distribution activities.

   2. Confirm logs include identifiers for correlating model development actions across training sessions, research activities, and model deployment events.

   3. Ensure structured formats (e.g., JSON, syslog) are used for consistency across model development and distribution logging systems.

   4. Check completeness of AI model log records by sampling training workflows, model access trails, and distribution activities.

   5. Validate that custom model events are logged where relevant (e.g., unauthorized model access attempts, training data poisoning detection, model parameter extraction).

   6. Review AI model audit logs for evidence of tampering or missing entries related to model training, research activities, and intellectual property access.

   7. Examine AI model audit records to ensure they contain relevant security information such as model training access, research data usage, model inference requests, and intellectual property protection events.

   8. Validate that AI model audit records include sufficient contextual information to support model security investigations, research reproducibility, and intellectual property forensics.

   9. Confirm that AI model audit record generation covers all security-relevant events across model development environments, training infrastructure, and distribution platforms.

   10. Review AI model audit record retention and storage mechanisms to ensure model security information remains available for intellectual property protection and research integrity timeframes.

   11. Verify log records include inference inputs, latency, and output summaries where feasible.

   12. Confirm logs capture model loading, version switching, and failures.

   13. Validate security-relevant events such as unauthorized access attempts.

   14. Ensure logs are tied to specific model versions and deployments.

   15. Check for structured logging practices to enable ML pipeline tracing.

   16. Review whether critical decisions based on models are traceable through logs.

LOG-10: Audit Records Protection

Control Specification

Protect audit records from unauthorized access, modification, and deletion.

Auditing Guidelines for Model Providers (MP)

1. Inquiring with Control Owners

   1. Conduct interviews with personnel responsible for protecting model development and training audit records from unauthorized access, modification, and deletion. Verify their understanding of security controls, access restrictions, monitoring procedures, and incident response processes related to audit records.

2. Inspecting Records and Documents

   1. Verify access controls enforce least-privilege and role-based restrictions for model-related audit records.

   2. Verify audit records are protected against unauthorized modification or deletion through tamper-resistant or immutable storage mechanisms.

   3. Review documentation demonstrating encryption of audit records at rest and during transmission.

   4. Verify audit records are segregated from operational data and protected independently.

   5. Review audit trails documenting access to audit record storage locations.

   6. Verify monitoring and alerting mechanisms detect unauthorized access, modification, or deletion attempts involving audit records.

   7. Examine backup and recovery procedures ensuring protection extends to archived audit records.

   8. Verify periodic testing and review of audit record protection controls.

   9. Review incident response procedures addressing compromise or suspected compromise of audit records.

LOG-11: Encryption Monitoring and Reporting

Control Specification

Establish and maintain a monitoring and internal reporting capability over the operations of cryptographic, encryption and key management policies, processes, procedures, and controls.

Auditing Guidelines for Model Providers (MP)

1. Inquiring with Control Owners

   1. Conduct interviews with personnel responsible for establishing and maintaining monitoring and internal reporting capabilities over cryptographic, encryption, and key management operations for AI model development and distribution to understand their oversight processes for intellectual property protection and model security. Verify their understanding of monitoring controls for encryption of model data, internal reporting mechanisms for model cryptographic operations, and procedures for maintaining ongoing oversight of key management for model protection and research data security.

2. Inspecting Records and Documents

   1. Confirm monitoring mechanisms are in place to detect encryption failures or unauthorized decryption attempts for AI model data, training datasets, and model parameters.

   2. Verify reports are generated on the use of encryption in AI model data transmission, model storage, and research data protection.

   3. Review documentation on how cryptographic keys are handled, rotated, and monitored for AI model protection, training data security, and intellectual property safeguarding.

   4. Validate that AI model teams receive alerts for deviations in encryption policy adherence affecting model security and research data protection.

   5. Check integration with central SIEM tools for real-time visibility into AI model cryptographic operations and intellectual property protection events.

   6. Ensure audit logs capture AI model encryption-related events like certificate expiration, invalid key use, or model data encryption failures.

   7. Confirm documentation of encryption algorithms and configurations in use for AI model storage, training data protection, and model distribution security.

   8. Examine internal reporting processes for communicating AI model cryptographic and key management findings to research leadership and intellectual property protection teams.

   9. Review periodic assessment and reporting schedules for AI model cryptographic policy compliance and intellectual property protection effectiveness.

   10. Validate that monitoring and reporting capabilities cover all aspects of AI model cryptographic operations including research data protection, model security, and intellectual property compliance.

   11. Confirm that ML pipeline stages are monitored for encryption status of data and model artifacts.

   12. Review logging of encryption usage when storing models, datasets, and inference results.

   13. Ensure key access for decrypting sensitive training datasets is logged and reviewed.

   14. Validate reporting includes usage of encryption for APIs serving the model.

   15. Verify that incidents of encryption bypass or misconfigurations are escalated and remediated.

   16. Check use of Hardware Security Modules (HSMs) or KMS in sensitive model operations is monitored.

LOG-12: Transaction/Activity Logging

Control Specification

Log and monitor key lifecycle management events to enable auditing and reporting on usage of cryptographic keys.

Auditing Guidelines for Model Providers (MP)

1. Inquiring with Control Owners

   1. Conduct interviews with personnel responsible for logging and monitoring key lifecycle management events for AI model operations to understand their processes for capturing, analyzing, and reporting on cryptographic key usage for model protection and research data security. Verify their understanding of key lifecycle event logging requirements for AI model development, monitoring procedures for encryption keys protecting intellectual property, and reporting capabilities that enable auditing and compliance oversight of cryptographic key management activities in model training and distribution.

2. Inspecting Records and Documents

   1. Verify that cryptographic key usage for AI model data encryption, training dataset protection, and model parameter security is logged by the model management systems.

   2. Confirm logs include timestamped records of key creation, use, rotation, and destruction for AI model storage, research data protection, and intellectual property safeguarding operations.

   3. Ensure visibility into key usage by different AI model development components, training infrastructure services, and model distribution systems.

   4. Validate alerts are generated on suspicious or unauthorized key operations affecting AI model security or research data protection.

   5. Check alignment with internal policy for lifecycle monitoring of keys used within AI model operations for intellectual property protection and research integrity.

   6. Review SIEM or monitoring tool integrations that centralize and analyze AI model key-related activities and intellectual property protection events.

   7. Confirm audit trails exist for every critical key management operation supporting AI model development, training, and distribution functionality.

   8. Examine reporting capabilities and procedures for generating AI model key lifecycle management reports to support intellectual property compliance and research security auditing requirements.

   9. Review log retention policies and practices to ensure AI model key lifecycle event records are maintained for intellectual property protection and research integrity timeframes.

   10. Validate that key lifecycle monitoring covers all AI model cryptographic operations including training data encryption, model parameter protection, and research backup security activities.

   11. Verify that all encryption keys used for model artifact protection are logged during lifecycle events.

   12. Confirm traceability of key usage in model training, inference, and storage.

   13. Ensure key access by developers, data scientists, or automation tools is logged and auditable.

   14. Validate anomaly detection is enabled for irregular key usage patterns in model pipelines.

   15. Check logs contain metadata linking key use to specific model or dataset identifiers.

   16. Confirm logs support forensic analysis of access to sensitive model output or weights.

LOG-13: Access Control Logs

Control Specification

Monitor and log physical access using an auditable access control system.

Auditing Guidelines for Model Providers (MP)

1. Inquiring with Control Owners

   1. Conduct interviews with personnel responsible for monitoring and logging physical access using auditable access control systems for AI model development and research facilities to understand their processes for tracking, recording, and reviewing physical access to areas containing model training infrastructure and intellectual property. Verify their understanding of access control system monitoring capabilities for model development environments, logging procedures for physical access to research facilities and training clusters, and audit trail requirements that ensure all physical access activities affecting model security and intellectual property protection are properly documented and reviewable.

2. Inspecting Records and Documents

   1. Verify physical access control systems are in place for all AI model development environments, including research facilities, training infrastructure, model storage, and distribution centers.

   2. Check logging mechanisms capture physical access timestamps, user identity, and location for AI model development facilities and training infrastructure.

   3. Confirm physical access logs are retained in accordance with intellectual property protection and research integrity requirements for AI model operations.

   4. Validate alerts are generated for unauthorized or after-hours physical access to AI model development facilities and training infrastructure.

   5. Review role-based access controls to ensure only authorized research personnel can retrieve physical access logs for AI model development environments.

   6. Confirm periodic audits assess physical access adherence across all AI model development facilities and intellectual property storage areas.

   7. Examine whether physical access logs are integrated into centralized SIEM systems for correlation with AI model security and research integrity events.

   8. Verify encryption is applied to stored physical access logs for AI model development facilities.

   9. Review monitoring procedures and capabilities of the physical access control system to ensure real-time visibility into physical access events affecting AI model development infrastructure.

   10. Validate that the access control system provides comprehensive audit trails with tamper-evident logging for all physical access activities in AI model development and research facilities.

   11. Examine backup and recovery procedures for AI model physical access control system logs to ensure continuity of audit capabilities for intellectual property protection compliance.

   12. Validate physical access logs track access to environments used for training and hosting AI models.

   13. Ensure physical access logs specifically tag access to GPU/TPU compute resources.

   14. Confirm logging configurations detect physical access to air-gapped model zones, if any.

   15. Ensure sensitive model stores (e.g., embedding vectors, checkpoints) have physical access coverage.

   16. Verify data center physical access logs can correlate to model access events.

   17. Check for restricted physical access to AI model tuning environments.

LOG-14: Failures and Anomalies Reporting

Control Specification

Define, implement and evaluate processes, procedures and technical measures for the reporting of anomalies and failures of the monitoring system and provide immediate notification to the accountable party.

Auditing Guidelines for Model Providers (MP)

1. Inquiring with Control Owners

   1. Conduct interviews with personnel responsible for defining, implementing, and evaluating processes for reporting AI model monitoring system anomalies and failures to understand their procedures for detecting, classifying, and immediately notifying accountable parties of model development and distribution issues affecting research integrity and intellectual property protection. Verify their understanding of technical measures for AI model anomaly detection, notification workflows for different model failure types, and evaluation processes that ensure AI model monitoring system reliability and timely escalation to responsible stakeholders including research leadership and model safety teams.

2. Inspecting Records and Documents

   1. Verify AI model systems are configured to detect logging anomalies such as dropped training events, model performance degradation, or research data format corruption affecting model integrity and intellectual property protection.

   2. Check processes are in place for classifying AI model failure severity and identifying responsible owners including research teams, model safety officers, and intellectual property protection staff.

   3. Validate AI model failures trigger alert workflows in ticketing or incident response platforms with appropriate escalation to research leadership and model safety teams.

   4. Ensure fallback mechanisms exist when primary AI model logging systems fail, including backup training monitoring and model performance tracking capabilities.

   5. Confirm logs of AI model failure events are themselves collected and analyzed to understand impact on research integrity and model security.

   6. Check that post-incident reviews incorporate root cause analysis for AI model failures with focus on research impact and intellectual property protection.

   7. Verify metrics are defined to track detection and resolution of AI model anomalies including research productivity impact and model security measures.

   8. Examine immediate notification procedures for AI model monitoring failures to ensure accountable parties including research directors and model safety teams receive timely alerts.

   9. Review evaluation processes for assessing the effectiveness of AI model anomaly reporting and failure notification procedures in maintaining research integrity and model security.

   10. Validate that technical measures include automated escalation mechanisms for AI model monitoring failures when initial notifications are not acknowledged by responsible research and safety teams.

   11. Verify failure detection covers both training logs and inference logs.

   12. Ensure anomalies include missing audit trails for fine-tuning or retraining events.

   13. Validate alerts are raised when model telemetry logging drops or fails.

   14. Confirm controls exist to detect shadow modifications in model performance metrics.

   15. Review incident reports to ensure failures tied to model access or drift are logged.

   16. Check feedback loops from anomaly detection influence model governance.

LOG-15: Input Monitoring

Control Specification

Log and monitor all input events (content and metadata) to enable auditing and reporting on the usage of AI models.

Auditing Guidelines for Model Providers (MP)

1. Inquiring with Control Owners

   1. Conduct interviews with personnel responsible for logging and monitoring all input events (content and metadata) to AI models to understand their processes for capturing, storing, and analyzing model input data for auditing model usage and reporting on inference patterns. Verify their understanding of input event logging requirements for AI model operations, monitoring procedures for model performance and usage analytics, and reporting capabilities that enable comprehensive auditing of AI model interactions and research insights.

2. Inspecting Records and Documents

   1. Confirm input logging covers all AI model access points including API endpoints, research interfaces, model serving platforms, and third-party model integrations.

   2. Verify logs capture request identity, source application, timestamp, model version, and input payload structure for AI model inference requests.

   3. Check that logging does not capture sensitive research data or proprietary information unless explicitly required for model improvement and properly protected.

   4. Validate logging covers both direct model API calls and indirect model usage through downstream applications and research platforms.

   5. Confirm that AI model logs are used to detect adversarial inputs, model abuse patterns, or malformed requests affecting model security.

   6. Review retention settings to ensure AI model input logs are stored in alignment with research requirements and intellectual property protection policies.

   7. Ensure AI model input logs feed into usage analytics dashboards for model performance monitoring and research insights.

   8. Verify access to AI model input logs is role-restricted to authorized researchers and fully auditable for intellectual property protection.

   9. Examine monitoring capabilities to ensure real-time visibility into AI model usage patterns, performance metrics, and research utilization trends.

   10. Validate that metadata logging includes comprehensive model context such as inference parameters, model configuration, research project details, and performance metrics.

   11. Review reporting mechanisms to confirm they provide adequate audit trails for AI model governance, research integrity, and intellectual property compliance.

   12. Verify logging captures input prompts, formats, and schema details sent to models.

   13. Confirm logs support auditability for model performance and bias analysis.

   14. Ensure guardrails exist to avoid storing sensitive input content unencrypted.

   15. Validate tagging of input data by dataset version or model version.

   16. Check logging of retraining input events for traceability.

   17. Confirm access control policies are applied to input logs across dev/test/prod.

   18. Ensure input logs are available for forensics during AI model misbehavior incidents.

LOG-16: Output Monitoring

Control Specification

Log and monitor all output events (content and metadata) to enable auditing and reporting on usage of AI models.

Auditing Guidelines for Model Providers (MP)

1. Inquiring with Control Owners

   1. Conduct interviews with personnel responsible for logging and monitoring all output events (content and metadata) from AI models to understand their processes for capturing, storing, and analyzing model-generated responses for auditing model performance and reporting on inference usage patterns. Verify their understanding of output event logging requirements for AI model operations, monitoring procedures for model response quality and safety, and reporting capabilities that enable comprehensive auditing of AI model outputs and research analytics.

2. Inspecting Records and Documents

   1. Verify logs capture AI model outputs returned to API consumers, research platforms, or downstream applications.

   2. Check logs include confidence scores, model version, timestamp, request ID, and inference parameters for AI model responses.

   3. Ensure auditability of AI model outputs that were flagged, filtered, or modified by safety systems or research protocols.

   4. Validate logs help detect model hallucinations, biased outputs, safety violations, or anomalous model responses affecting research integrity.

   5. Confirm AI model outputs are categorized by risk level, content type, and research classification for review prioritization and safety assessment.

   6. Ensure AI model output logs are used to train safety filters, bias detection systems, or model improvement processes.

   7. Verify access to AI model output logs is restricted to authorized researchers and properly encrypted for intellectual property protection.

   8. Confirm AI model output monitoring feeds into usage analytics for model performance insights and research utilization trends.

   9. Examine reporting mechanisms for AI model outputs to ensure they provide comprehensive audit trails for model safety compliance and research governance oversight.

   10. Validate that metadata logging includes complete model context such as inference parameters, training version details, research project information, and output quality metrics.

   11. Review AI model output log retention policies to ensure compliance with research requirements and enable long-term auditing of model behavior and performance patterns.

   12. Confirm model-generated output logs include prompt-response pairs with tags for context.

   13. Validate outputs are categorized (e.g., informative, rejected, risky) and stored accordingly.

   14. Ensure output logs are available for post-hoc fairness and bias assessments.

   15. Verify logs contribute to explainability and auditability dashboards.

   16. Check for redaction or summarization features in logs involving long-form outputs.

   17. Ensure logs indicate usage of specific tuning datasets or chain-of-thought parameters.

   18. Validate monitoring tracks drift in output formats over time.

MDS: Model Security

MDS-01: Training Pipeline Security

Control Specification

Define, implement, and evaluate policies, procedures, and technical measures that ensure the security of the Training Pipeline. Regularly review and update policies, procedures and technical measures to address new security threats and best practices.

Auditing Guidelines for Model Providers (MP)

Policy and Procedure Review: Review documented processes and procedures for the entire AI training pipeline, covering all stages (data preprocessing, model training, integration), security of datasets and pre-trained models, and codebase security, ensuring formal approval of these policies and procedures.

Best Practices Adherence: Assess alignment with security best practices (e.g., OWASP Top 10).

1. Implementation Verification: Audit implementation through extensive reviews, including deep code reviews, examination of data handling practices, assessment of training infrastructure security, and interviews with relevant personnel.

2. Regular Review and Update: Review evidence of regular reviews and updates.

3. Coordination with OSP and/or AP: Verify clear definition/documentation of responsibilities and assess communication/coordination.

4. Data and Pre-trained Model Security (if applicable): Verify how datasets are being used and secured. Verify how pre-trained models are being used and secured.

MDS-02: Model Artifact Scanning

Control Specification

Define, implement, and evaluate policies, procedures, and technical measures for the scanning of model artifacts for vulnerabilities and attacks, at each step of the service lifecycle and at each hand over point. Regularly review and update policies, procedures and technical measures to address model artifact scanning.

Auditing Guidelines for Model Providers (MP)

1. Examine documented processes/procedures for periodic scanning of all model artifacts, ensuring coverage across the entire model lifecycle.

2. Assess if scanning frequency aligns with risk appetite/best practices.

3. Verify appropriate scanning tool usage.

4. Inspect configurations and reports.

5. Confirm vulnerability remediation.

6. Check for CI/CD pipeline scanning integration.

7. Verify a documented process for secure model hand-over.

8. Examine evidence of regular process, procedure, and tool reviews/updates.

9. Confirm communication of vulnerability information and remediation steps to OSP and AP (as applicable).

MDS-03: Model Documentation

Control Specification

Define, implement, enforce, approve, document, communicate, maintain and evaluate processes and procedures for model documentation. Regularly review and update the model documentation.

Auditing Guidelines for Model Providers (MP)

1. Review the MP’s defined processes for creating standardized model documentation, such as Model Cards. This should include information about model details, intended uses, limitations, and performance metrics.

2. Verify the existence and application of a formal approval process for all model documentation before it is finalized and communicated.

3. Audit the implementation of the documentation process by examining a sample of actual Model Cards for completeness, accuracy, and consistency.

4. Assess the MP’s procedures for evaluating the ongoing accuracy of model documentation and maintaining it throughout the model lifecycle, including version control, such as data versioning for different iterations of a model.

5. Check defined methods for communicating model documentation to relevant stakeholders (e.g., OSP, AP, internal teams).

6. Verify procedures for the secure hand-over of model documentation.

7. Review processes that track if documentation is created or is being used by team members.

MDS-04: Model Documentation Requirements

Control Specification

Establish and implement baseline requirements for Model documentation.

Auditing Guidelines for Model Providers (MP)

1. Review that there are processes in place for the development and documentation of standardized attributes for each model, and review the standards and policies.

2. Evaluate with leadership whether the Model Card can be followed at all lifecycle stages of the model.

3. Assess the process and policy and evaluate the scope against the risk management plan.

4. Audit the validation of the data that must occur on an annual basis.

MDS-05: Model Documentation Validation

Control Specification

Define, implement, and evaluate processes, procedures, and technical measures for the validation of the Model documentation aligned with the current model.

Auditing Guidelines for Model Providers (MP)

1. Review the procedures used for validation to check if there was an update or a lifecycle event, and determine the role in documentation validation in the service.

2. Evaluate and assess the framework in use with regards to accuracy and controls within the Model/System Card to ensure Model/System Card consistency with the Model implementation.

3. Check whether the validation activity includes both model performance and ethical/safety-related characteristics in accordance with the standards for AI.

4. Inspect and document validation, including its scope, test environment and success/failure criteria, and verify its test and development procedures with security requirements, as specified in an established policy.

5. Assess the criteria of success/failure for ongoing activities, and examine records of formal communication, such as alerts or dashboards.

MDS-06: Adversarial Attack Analysis

Control Specification

Define, implement, and evaluate processes and technical measures to assess adversarial threats specific to each AI model.

Auditing Guidelines for Model Providers (MP)

1. Examine documented processes and technical measures for conducting model-specific adversarial threat assessments.

2. Verify that adversarial attack analysis is performed for each AI model, with explicit consideration of model architecture, training data characteristics, and intended use cases.

3. Review methodologies for identifying model-specific vulnerabilities, including data poisoning opportunities, prompt injection risks, extraction vulnerabilities, and evasion techniques.

4. Assess the threat prioritization framework, verifying that attack vectors are ranked based on technical feasibility, potential impact, and likelihood.

5. Examine documentation of adversarial threat models for each AI system, confirming completeness and technical accuracy.

6. Verify implementation of testing protocols that simulate high-priority attack vectors, including red team exercises and technical adversarial testing.

7. Review processes for updating threat assessments when models change or new attack techniques emerge.

8. Assess mechanisms for sharing threat insights with downstream consumers (OSPs, APs, AICs).

9. Verify that adversarial threat assessments influence security controls implemented in model development.

MDS-07: Robustness against Adversarial Attack / Model Hardening

Control Specification

Define, implement, and evaluate processes, procedures, and technical measures for Model Hardening to mitigate relevant adversarial attacks as identified in the Threat Analysis and Adversarial Threat Analysis.

Auditing Guidelines for Model Providers (MP)

1. Examine the MP’s documented approach to model hardening that addresses threats identified through adversarial threat analysis (MDS-06).

2. Verify implementation of specific model hardening techniques appropriate to the model architecture and identified threats.

3. Assess documentation of hardening testing procedures and results, confirming that techniques effectively mitigate identified vulnerabilities.

4. Review the periodic re-evaluation process for hardening measures to adapt to emerging threats.

5. Verify that model hardening measures are implemented without unduly impacting model performance, and any performance tradeoffs are documented and approved.

6. Confirm that model hardening measures are thoroughly documented and communicated to OSPs and APs to enable appropriate downstream implementation.

MDS-08: Model Integrity Checks

Control Specification

Regularly calculate and compare checksums using cryptographic hashes of model checkpoints to detect unauthorized modifications. Apply at least annually based on the level of risk, or after any change of hands.

Auditing Guidelines for Model Providers (MP)

1. Examine the documented process for generating cryptographic checksums of model checkpoints and versions.

2. Verify that strong cryptographic algorithms are used for checksum generation.

3. Assess the security of the checksum repository, confirming appropriate access controls and backup procedures.

4. Review procedures for regular (at least annual) verification of model integrity against baseline checksums.

5. Examine the process for securely communicating checksums to downstream consumers.

6. Verify documentation of baseline checksums for all model versions in the release history.

7. Confirm that integrity verification occurs after any model transfer or change of hands.

MDS-09: Model Signing/Ownership Verification

Control Specification

Sign models cryptographically and verify signatures to ensure model provenance and ownership, any time the model changes hands or is loaded from storage.

Auditing Guidelines for Model Providers (MP)

1. Examine the documented model signing process, verifying the use of industry-standard cryptographic algorithms.

2. Assess the security controls protecting private signing keys, including access restrictions and key rotation procedures.

3. Verify that all model versions and checkpoints are cryptographically signed before release.

4. Review the process for securely distributing public keys or certificates to downstream consumers.

5. Confirm documentation of the signing algorithms, key strengths, and certification authorities used.

6. Verify implementation of signing verification prior to any deployment or distribution of models.

MDS-10: Model Continuous Monitoring

Control Specification

Define, implement, and evaluate processes, procedures, and technical measures for continuous monitoring of model performance metrics over time to identify sudden shifts or unexpected changes in predictions that could degrade model performance.

Auditing Guidelines for Model Providers (MP)

1. Examine the MP’s documented approach to continuous model performance monitoring.

2. Verify that appropriate performance metrics are defined based on model type and intended use cases.

3. Assess the establishment of performance baselines and acceptable thresholds for each metric.

4. Review anomaly detection algorithms implemented to identify unexpected shifts in model behavior.

5. Verify the frequency and comprehensiveness of performance monitoring reports.

6. Confirm that monitoring capabilities integrate with alerting systems to enable timely response to anomalies.

7. Examine processes for investigating and remediating detected performance degradation.

MDS-11: Model Failure

Control Specification

Perform a risk-based evaluation of the model and model serving infrastructure for model failure. Define and implement measures to mitigate model and model serving infrastructure failures, and regularly evaluate throughout the AI system’s lifecycle.

Auditing Guidelines for Model Providers (MP)

1. Examine documentation of model characteristics relevant to redundancy planning.

2. Verify that models are designed to function effectively in redundant deployments.

3. Assess whether multiple model variants or versions are available to support diversity in redundancy.

4. Review compatibility with backup and restoration processes.

5. Examine implementation of architectures providing inherent redundancy (e.g., mixture of experts, ensemble models).

6. Confirm that model design accounts for potential failure modes and mitigations.

7. Verify testing procedures that validate model behavior in redundant configurations.

MDS-12: Open Model Risk Assessment

Control Specification

Establish a process to evaluate risk associated with open models. Periodically review these risk factors, and implement a process to monitor and mitigate any determined vulnerabilities.

Auditing Guidelines for Model Providers (MP)

1. Examine documented risk assessment methodology specific to open weight models.

2. Verify comprehensive identification and cataloging of potential vulnerabilities associated with exposed model weights.

3. Review implementation of model-level mitigations such as weight obfuscation, defensive distillation, or architectural hardening.

4. Assess the process for regularly reviewing and updating risk assessments as new vulnerabilities emerge.

5. Verify that clear guidance is provided to downstream consumers on secure usage of open weight models.

6. Confirm that security testing includes scenarios specific to open weight model vulnerabilities.

MDS-13: Secure Model Format

Control Specification

Adopt secure model formats and processes for AI model serialization where applicable.

Auditing Guidelines for Model Providers (MP)

Guidelines focus on the OSP’s responsibility to maintain serialization security during orchestration:

1. Format documentation verification ensures transparency about supported serialization approaches.

2. Validation implementation assessment addresses protections when loading external models.

3. Conversion security review addresses the critical risk point of format transformations.

4. Lifecycle security verification ensures consistent protection throughout orchestration processes.

5. MP guideline adherence confirmation ensures alignment with model-specific security requirements.

6. Communication verification addresses proper knowledge transfer to downstream consumers.

SEF: Security Incident Management, E-Discovery, & Cloud Forensics

SEF-01: Security Incident Management Policy and Procedures

Control Specification

Establish, document, approve, communicate, apply, evaluate and maintain policies and procedures for Security Incident Management, E-Discovery, and Forensics. Review and update the policies and procedures at least annually, or upon significant changes.

Auditing Guidelines for Model Providers (MP)

1. Verify the Model Provider (MP) has a documented and approved security incident management policy, aligned with recognized industry standards such as NIST SP 800-61, NIST AI.100 or ISO/IEC 27035.

2. Confirm that roles and responsibilities for incident detection, reporting, escalation, and resolution are clearly defined and documented. Ensure coordination points are identified with other parties (e.g., such as OSPs, APs, AICs).

3. Ensure the Model Provider (MP) has procedures for E-Discovery and Forensics for AI Model and use.

4. Check that procedures cover the full incident lifecycle, including initial reporting, triage, escalation criteria, containment, eradication, recovery, and post-incident review.

5. Ensure that the policy and procedures are communicated effectively to all internal and external stakeholders, including third-party service providers, where applicable.

6. Verify that the incident management policy and related procedures are reviewed and updated periodically, or following major incidents, organizational changes, or regulatory updates.

7. Confirm that regular training is provided to incident response teams, with materials updated based on emerging threats and lessons learned.

8. Validate that incident response drills or tabletop exercises are conducted regularly, with documentation of scenarios, participants, outcomes, and improvement actions.

SEF-02: Service Management Policy and Procedures

Control Specification

Establish, document, approve, communicate, apply, evaluate and maintain policies and procedures for the timely management of security incidents. Review and update the policies and procedures at least annually, or upon significant changes.

Auditing Guidelines for Model Providers (MP)

1. Confirm the AP has documented policies and procedures to ensure timely response of incidents.

2. Verify timely management expectations have been established and are based on business needs (e.g., regulations, contracts, incident severity level, ability to retain data).

3. Review dependencies and partners (e.g., AP, OSP) which could impact the ability of the MP to respond to the planned timelines.

4. Confirm regular audits of service management effectiveness and timely response to incidents.

5. Validate audit findings and lessons learned are addressed.

6. Verify documented training provided for service management procedures.

SEF-03: Incident Response Plans

Control Specification

Establish, document, approve, communicate, apply, evaluate and maintain a security incident response plan, which includes but is not limited to: a communication strategy for notifying relevant internal departments, impacted service customers, and other business critical relationships (such as supply-chain) that may be impacted.

Auditing Guidelines for Model Providers (MP)

1. Verify MP has incident response plans clearly documented and approved.

2. Confirm incident response plans cover critical scenarios for executing the model services comprehensively.

3. Check plans define specific roles and escalation procedures.

4. Verify the incident response plan includes a documented communication strategy for notifying internal functions, affected service customers, and upstream or downstream dependencies related to model development, hosting, or deployment.

5. Ensure regular reviews and updates of incident response documentation.

6. Confirm testing and drills of incident response plans performed periodically.

7. Verify documented corrective actions following response plan testing.

SEF-04: Incident Response Testing

Control Specification

Exercise the incident response plans at planned intervals or upon significant changes.

Auditing Guidelines for Model Providers (MP)

1. Confirm the MP executes incident response exercises at scheduled intervals or after significant changes to model architecture, training data sources, or deployment context.

2. Verify exercise records capture objectives, execution details, identified gaps, and follow-up actions.

3. Review how response plans are refined based on exercise results, especially in model integrity or poisoning scenarios.

4. Ensure relevant participants (e.g., AI engineers, data scientists, compliance officers) are involved.

5. Validate exercises reflect model-specific risks such as prompt injection, adversarial inference, or supply chain compromise.

6. Confirm outputs from exercises are evaluated and incorporated into formal response plan updates.

SEF-05: Incident Response Metrics

Control Specification

Establish, monitor and report information security incident metrics.

Auditing Guidelines for Model Providers (MP)

1. Verify documented metrics for evaluating incident response effectiveness.

2. Confirm metrics align with model parameters, organizational goals and industry best practices.

3. Check regular collection, analysis, and reporting of response metrics.

4. Ensure documentation of actions taken based on metrics analysis.

5. Confirm clear accountability for monitoring incident response metrics.

SEF-06: Event Triage Processes

Control Specification

Define, implement and evaluate processes, procedures and technical measures supporting business processes to triage security-related events.

Auditing Guidelines for Model Providers (MP)

1. Verify MP has documented triage procedures clearly define event categorization and prioritization.

2. Confirm triage processes efficiently differentiate between critical and non-critical events.

3. Confirm design supports information collection to support triage (e.g., model performance, data flow, model configuration).

4. Understand triage models from suppliers and partners (e.g., OSP, CSP, AP, AIC).

5. Check regular training provided on event triage methods, for the model.

6. Ensure continuous improvement through periodic review and update of triage processes.

7. Confirm clear accountability assigned for triaging security events.

SEF-07: Incident Management and Response

Control Specification

Define, implement and evaluate processes, procedures and technical measures for timely and effective response to security incidents in accordance with incident categories and severity levels. Review, update, and test processes and procedures at least annually.

Auditing Guidelines for Model Providers (MP)

1. Verify incident response categories and severity levels clearly documented (e.g., consider impacts such as model failure, agent misbehavior, bias/fairness violation, model drift).

2. Verify well defined response procedures are determined, including automated responses where applicable (e.g., model monitoring and alerts; model anomaly automated response; integration with application, orchestration, cloud service provider and/or AIC).

3. Confirm well-defined roles and escalation pathways during incident response.

4. Check documented incident response timelines and service level agreements (SLAs).

5. Ensure regular reviews of incident response activities and outcomes. Maintain documented records of reviews and outcomes, and document actions taken for any deviations observed

6. Verify processes and procedures for incident handling are formally defined and tested at least annually or bi-annually.

7. Confirm training is provided to relevant stakeholders on incident response processes at a frequency defined in alignment with organizational policies.

SEF-08: Security Breach Notification

Control Specification

Define and implement processes, procedures and technical measures for security breach notifications. Report material security breaches including any relevant supply chain breaches, as per applicable SLAs, laws and regulations.

Auditing Guidelines for Model Providers (MP)

1. Verify the MP has documented policies clearly specify requirements for breach notification.

2. Confirm procedures comply with applicable legal and regulatory requirements.

3. Confirm the notification procedure covers essential information (e.g., model impacted, model data impacted, output from model).

4. Ensure regular testing of breach notification procedures.

5. Verify reporting of breaches to appropriate parties within defined SLA and confirm procedures are reviewed on a periodic basis to incorporate changes in laws and SLA requirements.

SEF-09: Incident Records Management

Control Specification

Establish and maintain a secure repository of security incident records. Regularly review the incident records to identify patterns, root causes, and systemic vulnerabilities, and implement relevant corrective measures.

Auditing Guidelines for Model Providers (MP)

1. Verify the Model Provider (MP) has documented policies that clearly specify requirements for collecting, classifying, storing, protecting and retaining incident records specific to the AI model lifecycle.

2. Verify the policies define clear trigger conditions for when a security incident must be recorded.

3. Confirm the MP maintains a secure incident record repository with appropriate access controls, encryption (in transit and at rest) and audit logging to prevent unauthorized access, modification or deletion.

4. Determine whether the MP conducts periodic reviews of incident records to identify recurring patterns, root causes and systemic vulnerabilities (e.g. repeated hallucination patterns, systematic bias amplification, training data weaknesses, measurable degradation of model performance or behavior over time) and whether the review cadence and review process are formally documented.

5. Confirm corrective actions identified through incident record analysis are documented, tracked, implemented and verified for effectiveness in addressing the identified issues.

6. Ensure records of reviews and corrective actions are retained and available for audit.

SEF-10: Points of Contact Maintenance

Control Specification

Maintain points of contact for applicable regulation authorities, national and local law enforcement, and other legal jurisdictional authorities. Review and update the points of contact at least annually.

Auditing Guidelines for Model Providers (MP)

1. Verify documented procedures for Model Providers (MP) to meet regulatory responsibilities and maintain points of contact.

2. Verify procedures for review of dependencies with OSP, AP, AIC and CSP that would impact Application Providers ability to meet its regulatory contact obligations (e.g., AI and data regulations)

3. Confirm regular updates and validation of points of contact.

4. Check records clearly document responsibility for points of contact maintenance.

5. Ensure immediate updates to contact information upon role changes.

6. Confirm periodic audits validating the accuracy and availability of contacts.

STA: Supply Chain Management, Transparency, and Accountability

STA-01: Supply Chain Risk Management Policies and Procedures

Control Specification

Establish, document, approve, communicate, apply, evaluate, and maintain policies and procedures for supply chain risk management. Review and update the policies and procedures at least annually, or upon significant changes.

Auditing Guidelines for Model Providers (MP)

1. Request and review the policy document covering training data, ML frameworks, model components, annotation suppliers.

2. Verify approval by leadership (e.g., signature, date, version log).

3. Interview ML and MLOps teams about awareness, and request training or internal policy sharing evidence.

4. Assess procurement checklists, onboarding templates, or vetting forms for AI-specific suppliers.

5. Review how model input sources, training data, pre-trained models are risk-classified and reassessed.

6. Confirm change logs or version history reflect reviews or updates tied to incidents or supplier changes.

7. Confirm policy alignment with applicable external standards and regulations governing supply chain and data use.

8. Verify existence of metrics or internal reviews that evaluate whether supplier risk management processes are effective over time.

STA-02: SSRM Policy and Procedures

Control Specification

Establish, document, approve, communicate, apply, evaluate and maintain policies and procedures for the application of the Shared Security Responsibility Model (SSRM) within the organization. Review and update the policies and procedures at least annually, or upon significant changes.

Auditing Guidelines for Model Providers (MP)

1. Verify that the MP has defined and documented SSRM policies and procedures specifying responsibilities related to model development, training data security, model outputs, and intellectual property protection.

2. Confirm that SSRM clearly delineates responsibilities between the MP and third parties for dataset security, inference endpoint protection, model deployment, fine‑tuning, and consumer‑side responsibilities.

3. Validate compliance of SSRM policies with applicable AI regulations, ethical guidelines, and industry best practices.

4. Check for formal approval of SSRM policies and procedures by authorized leadership.

5. Ensure the SSRM has been communicated clearly to internal teams (e.g., RandD, deployment, legal) and external consumers (APs, OSPs) to prevent ambiguity in shared responsibilities.

6. Verify operational implementation of SSRM policies throughout the AI model lifecycle, including training, deployment, and updates.

7. Ensure the presence of metrics or periodic audits to evaluate SSRM effectiveness in ensuring model security, safety, and compliance.

8. Confirm periodic review and update of SSRM policies at least annually, or upon significant changes in technology, regulation, or the threat landscape.

STA-03: SSRM Supply Chain

Control Specification

Apply, document, implement and manage the SSRM throughout the supply chain.

Auditing Guidelines for Model Providers (MP)

1. Verify that the model provider discloses its SSRM in customer-facing materials, contracts, and documentation, clearly outlining shared responsibilities related to model development, deployment, and integration.

2. Evaluate how the model provider manages inherited responsibilities from upstream providers such as infrastructure from CSPs or orchestration layers and how these are incorporated into its own security and operational controls.

3. Review the structure and clarity of the responsibility matrix, ensuring it defines roles and obligations across the supply chain including CSPs, orchestration providers, application developers, and customers for transparency and accountability.

4. Verify that the MP has incorporated SSRM controls into day-to-day development and operational workflows, particularly where third-party tools, datasets, APIs, or pre-trained models are used. This includes ensuring shared responsibilities are explicitly acknowledged and addressed in integration processes, technical safeguards, and monitoring procedures.

STA-04: SSRM Guidance

Control Specification

Provide SSRM Guidance to the service customers detailing information about the SSRM applicability throughout the supply chain.

Auditing Guidelines for Model Providers (MP)

1. Verify the model provider offers clear and accessible SSRM guidance to AI service customers (AICs), outlining shared responsibilities related to model development, deployment, integration, and usage across the AI supply chain.

2. Review the model provider’s public documentation, trust center, or support resources for detailed information on service customer responsibilities such as securing model endpoints, managing API access, monitoring model behavior, and ensuring responsible use of model outputs.

3. Examine whether the MP provides SSRM guidance specifically addressing model-centric risks, such as training data provenance and bias, model explainability, adversarial robustness, and output misuse safeguards. Confirm that such guidance is shared with relevant downstream parties (e.g., APs, OSPs).

STA-05: SSRM Control Ownership

Control Specification

Delineate the shared ownership and applicability of all CSA AICM controls according to the SSRM.

Auditing Guidelines for Model Providers (MP)

1. Confirm the AI model provider has mapped all CSA AICM controls to its internal framework, identifying which are provider-owned, inherited (e.g., from CSP (e.g., IAM) or MP (e.g., encryption of model output)), or customer-owned (AIC), with clear documentation.

2. Review the mapping using SSRM guidance to ensure accuracy. Validate ownership assumptions, resolve gaps (e.g., unclear responsibility for model monitoring), and update regularly.

STA-06: SSRM Documentation Review

Control Specification

Review and validate the SSRM documentation.

Auditing Guidelines for Model Providers (MP)

1. Confirm the Model Provider (MP) has a process to regularly review its own SSRM documentation and that of key suppliers (e.g., CSPs, infrastructure providers), ensuring shared responsibilities are clearly defined and current by updating its matrix to reflect model-layer responsibilities such as model training, inference security, and API access control.

2. Examine these reviews are conducted at least annually or when major service changes occur (e.g., new model deployments, architecture changes, or API updates), ensuring the SSRM reflects changes in data flow, logging, and control ownership.

STA-07: SSRM Control Implementation

Control Specification

Implement, operate, and audit or assess the portions of the SSRM which the organization is responsible for.

Auditing Guidelines for Model Providers (MP)

1. Verify through evidence review that the MP is implementing its controls, such as securing the training pipeline, performing adversarial testing on the model, and providing a secure inference endpoint.

2. Review the MP’s security testing reports and model governance records.

STA-08: Supply Chain Inventory

Control Specification

Develop and maintain an inventory of all supply chain relationships.

Auditing Guidelines for Model Providers (MP)

1. Confirm that the model provider (MP) maintains a centralized and up-to-date inventory of all third-party relationships involved in model development, training, deployment, and maintenance.

2. Verify that all integrated service relationships such as data providers, compute infrastructure, model hosting platforms, and API integrations are accurately documented.

3. Determine whether the inventory is subject to regular review and validation to ensure its completeness, accuracy, and alignment with current operational, compliance, and model lifecycle requirements.

STA-09: Service Bill of Material (BOM)

Control Specification

Define, implement, and enforce a process for establishing a Bill of Material for the service supply chain. Review and update the Bill of Material at least annually, or upon significant changes.

Auditing Guidelines for Model Providers (MP)

1. Assess whether the model provider (MP) maintains a documented SBoM process, with regular reviews triggered by changes to model infrastructure, dependencies, or security posture.

2. Evaluate the completeness of the SBoM, ensuring it defines all relevant components such as APIs, model versions, scaling configurations, dependencies, security controls, and risk classifications with appropriate metadata.

3. Confirm inclusion of both infrastructure and AI-specific elements, such as compute resources, networking, model endpoints, training pipelines, and monitoring tools.

4. Inspect the security and access controls of the SBoM registry, verifying role-based access for authorized stakeholders, including cloud security teams and application integrators.

5. Review the timeliness and accuracy of SBoM updates following changes to model architecture, deployment configurations, or integrations, with documented impact assessments reviewed by security teams.

6. Verify that SBoM information is clearly communicated, including model capabilities (e.g., input/output formats, supported languages, latency benchmarks, accuracy metrics), SLAs, limitations, and integration protocols, with validated security and compliance disclosures.

STA-10: Supply Chain Risk Management

Control Specification

Periodically review risk factors associated with supply chain relationships.

Auditing Guidelines for Model Providers (MP)

1. Confirm the MP reviews supply chain risks at least annually or after major changes such as onboarding a new data provider or changes in model deployment environments addressing security, compliance, operational, and reputational risks.

2. Ensure risk assessments are documented, regularly updated, and informed by indicators like SLA breaches, security incidents, or audit findings (e.g., repeated model performance issues or encryption gaps while involving relevant internal teams).

3. Verify that identified risks result in mitigation actions such as updating contracts, enhancing oversight, or replacing vendors and that these actions are tracked and supported by audit-ready documentation, especially when providers fail to meet data privacy or performance standards.

STA-11: Primary Service and Contractual Agreement

Control Specification

Service agreements must incorporate at least the following mutually-agreed upon provisions and/or terms: • Scope, characteristics and location of business relationship and services offered • Information security requirements (including SSRM) • Change management process • Logging and monitoring capability • Incident management and communication procedures • Right to audit and third party assessment • Service termination • Interoperability and portability requirements • Data privacy • Operational Resilience

Auditing Guidelines for Model Providers (MP)

1. Examine that the model provider (MP) includes all critical provisions in service agreements with third parties, covering the scope of services, security requirements (including SSRM), change management, logging and monitoring, incident response, audit rights, service termination, interoperability, data privacy, and operational resilience.

2. Verify that the MP periodically reviews and enforces these contractual terms through audits, performance assessments, or compliance checks to ensure third-party adherence.

STA-12: Supply Chain Agreement Review

Control Specification

Review supply chain agreements at least annually, or upon significant changes.

Auditing Guidelines for Model Providers (MP)

1. Examine if the model provider reviews supply chain agreements with key supply chain partners such as CSPs, OSPs, APs least annually or when significant changes occur in services, risk, or regulations.

2. Verify that review outcomes are documented, and that identified risks or gaps are addressed through updated contracts, mitigation actions, or vendor reassessments, with oversight from governance or risk teams.

STA-13: Supply Chain Compliance Assessment

Control Specification

Define and implement a process for conducting internal assessments to confirm conformance and effectiveness of standards, policies, procedures, and service level agreement activities at least annually.

Auditing Guidelines for Model Providers (MP)

1. Establish whether the Model Provider maintains a recurring, structured audit process covering model development, training data governance, deployment infrastructure, and service delivery such as model APIs, version control, fine-tuning pipelines, and model monitoring systems.

2. Assess audit documentation for issues including data quality, labeling inconsistencies, model bias, performance drift, explainability gaps, or unauthorized access. Validate that corrective actions are tracked, resolved in a timely manner, and aligned with applicable AI standards, ethical guidelines, and internal controls.

3. Determine whether audit findings are communicated to relevant stakeholders—such as ML engineers, compliance teams, and product owners—and that a feedback mechanism exists to continuously refine audit practices, improve model lifecycle management, and support responsible AI deployment.

STA-14: Supply Chain Service Agreement Compliance

Control Specification

Implement policies requiring all service providers throughout the supply chain to comply with information security, confidentiality, access control, privacy, audit, personnel policy and service level requirements and standards.

Auditing Guidelines for Model Providers (MP)

1. Examine whether the model provider (MP) has defined and implemented a process for incorporating security, compliance, and governance requirements into contractual agreements with all relevant supply chain partners, including data providers, model hosting platforms, and infrastructure vendors.

2. Determine whether these requirements have been explicitly included in executed contracts, ensuring they cover areas such as data protection, model integrity, regulatory compliance, and service-level expectations.

3. Evaluate whether the model provider (MP) has preserved the right to audit or assess supply chain partners, where necessary, to verify adherence to contractual obligations and to mitigate risks related to sensitive data, AI model performance, and operational continuity.

STA-15: Supply Chain Governance Review

Control Specification

Review the organization’s service providers’ IT governance policies and procedures at least annually, or upon significant changes.

Auditing Guidelines for Model Providers (MP)

1. Examine whether the model provider (MP) has defined and implemented a process for reviewing the governance practices of its supply chain partners, including those involved in data provisioning, model training infrastructure, and inference deployment.

2. Evaluate whether the MP conducts these reviews at least annually, or upon significant changes, and maintains documented evidence of such reviews being performed in alignment with the process.

STA-16: Supply Chain Data Security Assessment

Control Specification

Define and implement a process for conducting risk-based security assessments of the supply chain.

Auditing Guidelines for Model Providers (MP)

1. Determine whether the MP has a process to assess supply chain data (risk-based) security, covering cloud infrastructure (e.g., IAM, storage) and AI components like training pipelines, third-party datasets, and inference APIs.

2. Examine how the MP identifies risks from data providers, cloud platforms, and model hosts, such as insecure transfers, outdated dependencies, or unclear data provenance, with defined roles across the AI lifecycle.

3. Assess whether documented procedures exist to manage risks from external entities involved in data supply, model hosting, or API integration, including access control, rate limiting, and vendor accountability.

4. Validate that regular security assessments are conducted per policy, addressing risks to data integrity (e.g., poisoning), model security (e.g., adversarial threats), and cloud infrastructure (e.g., misconfigurations).

TVM: Threat & Vulnerability Management

TVM-01: Threat and Vulnerability Management Policy and Procedures

Control Specification

Establish, document, approve, communicate, apply, evaluate and maintain policies and procedures to identify, report and prioritize the remediation of vulnerabilities and threats, in order to protect systems against vulnerability exploitation. Review and update the policies and procedures at least annually, or upon significant changes.

Auditing Guidelines for Model Providers (MP)

1. Verify that the MP has established and documented TVM policies and procedures defining scope, objectives, roles, and responsibilities.

2. Inspect whether the policies are compliant with regulatory requirements, industry best practices, and threat scenarios.

3. Verify formal approval of the policies by authorized management.

4. Verify communication of the policies to all relevant stakeholders and their understanding.

5. Confirm that the policies are applied in daily operations.

6. Verify that metrics and monitoring are in place to evaluate effectiveness and identify improvements.

7. Inspect evidence that the policies are reviewed and updated at least annually or upon significant changes.

8. Verify that the TVM policy explicitly covers the ML development environment, including frameworks (e.g., TensorFlow, PyTorch), open‑source libraries, and underlying infrastructure.

9. Confirm that the policy addresses AI‑specific threats, such as vulnerabilities in model serialization formats, inference servers, and ML pipelines.

10. Verify that policies and procedures include formalized mechanisms and dedicated communication channels for Vulnerability Disclosure activities.

TVM-02: Malware and Malicious Instructions Protection Policy and Procedures

Control Specification

Establish, document, approve, communicate, apply, evaluate and maintain policies and procedures to protect against malware and malicious instructions. Review and update the policies and procedures at least annually, or upon significant changes.

Auditing Guidelines for Model Providers (MP)

1. Verify that the Model Provider (MP) has established and documented policies and procedures in the domain of Malware Protection that, by defining organizational and technical measures to prevent, detect, examine and remove malicious codes from systems, aim at leading efforts to protect the latter against malware attacks. Ensure that the policies are documented in detail, covering scope, objectives, roles and responsibilities.

2. Inspect whether the above-mentioned policies and procedures are compliant with relevant regulatory requirements, industry best practices and the specific threat scenarios to which the organization is potentially exposed.

3. Verify that the above-mentioned policies and procedures have been formally approved by authorized parties (e.g., management sign-off).

4. Verify that the above-mentioned policies and procedures (in both their original and subsequent versions) have been adequately communicated by authorized parties to all relevant stakeholders and that their content has been thoroughly comprehended by them.

5. Confirm that the policy is concretely and appropriately applied by involved parties in their day-to-day operations.

6. Verify that metrics and Key Performance Indicators (KPIs) have been established and are continuously monitored to evaluate the effectiveness of the above-mentioned policies and procedures and identify possible improvement areas.

7. Inspect whether the above-mentioned policies and procedures are periodically reviewed and updated (at least annually) by responsible parties.

8. Verify the MP’s policy includes scanning of training datasets for malware or malicious content that could be used in a data poisoning attack.

9. Confirm the policy requires scanning of the ML development environment and tools for malware.

TVM-03: Vulnerability Identification

Control Specification

Define, implement and evaluate processes, procedures and technical measures for the detection of vulnerabilities on organizationally managed assets at least monthly.

Auditing Guidelines for Model Providers (MP)

1. Verify that vulnerability detection measures (e.g., scanning, agent-based monitoring) are implemented across all organizationally managed assets, and that scans or detection activities occur at least monthly.

2. Inspect whether the above-mentioned policies, procedures, and technical measures are compliant with relevant regulatory requirements and industry best practices.

3. Confirm that the above-mentioned policies, procedures, and technical measures are concretely and appropriately applied by involved parties in their day-to-day operations.

4. Inspect whether the above-mentioned policies, procedures, and technical measures are monitored against sets of efficacy and efficiency metrics / indicators.

5. Inspect whether the above-mentioned policies, procedures, and technical measures are periodically reviewed and updated by responsible parties.

6. Verify the MP’s remediation schedule covers vulnerabilities in the ML software supply chain.

7. Confirm a process exists for emergency responses to vulnerabilities that could allow model theft or manipulation, which may involve model retraining or redeployment.

TVM-04: Threat Analysis and Modelling

Control Specification

Define, implement and evaluate threat analysis processes and procedures to identify, assess and review the threat landscape for Cloud and AI systems. Build threat models according to industry best practices to inform the risk mitigation strategy.

Auditing Guidelines for Model Providers (MP)

1. Verify that the Model Provider (MP) has defined processes, procedures, and technical measures to systematically identify threats to which AI systems and models are potentially exposed. Ensure that the processes are documented in detail, covering scope, objectives, roles and responsibilities.

2. Verify that processes, procedures, and technical measures are in place to systematically assess threats to AI systems and models previously identified.

3. Inspect whether the above-mentioned processes, procedures, and technical measures of threat analysis are compliant with relevant regulatory requirements and industry best practices.

4. Verify that countermeasures against identified threats are timely defined, prioritized, accordingly applied, monitored, reviewed and updated by relevant parties.

5. Inspect whether the above-mentioned processes, procedures, and technical measures of threat analysis are monitored against sets of efficacy and efficiency metrics / indicators.

6. Inspect whether the above-mentioned processes, procedures, and technical measures of threat analysis are periodically reviewed and updated by responsible parties.

TVM-05: Detection Updates

Control Specification

Define, implement and evaluate processes, procedures and technical measures to update detection tools, threat signatures, and indicators of compromise on a weekly, or more frequent basis.

Auditing Guidelines for Model Providers (MP)

1. Verify that the Model Provider (MP) has defined processes, procedures, and technical measures to update tools implemented to detect vulnerabilities, threat signatures and indicators of compromise within the security perimeter at least weekly. Ensure that the processes are documented in detail, covering scope, objectives, roles and responsibilities.

2. Verify that the above-mentioned processes, procedures, and technical measures are compliant with relevant regulatory requirements and industry best practices.

3. Confirm that the above-mentioned processes, procedures, and technical measures are concretely and appropriately applied by involved parties in their day-to-day operations.

4. Inspect whether the above-mentioned processes, procedures, and technical measures are monitored against sets of industry-standard efficacy and efficiency metrics / indicators.

5. Inspect whether the above-mentioned policies, procedures, and technical measures are periodically reviewed and updated by responsible parties.

6. Verify the MP updates the detection rules and signatures for the tools monitoring its training environments and inference endpoints.

7. Confirm the process for updating its adversarial attack detection models or heuristics based on new research.

TVM-06: External Library Vulnerabilities

Control Specification

Define, implement and evaluate processes, procedures and technical measures to identify updates for applications which use third party or open source libraries according to the organization’s vulnerability management policy.

Auditing Guidelines for Model Providers (MP)

1. Verify that the Model Provider (MP) has defined processes, procedures, and technical measures to identify and implement updates for applications that use third party or open source libraries, in order to mitigate risks of compromise associated with the exploitation of vulnerabilities within such libraries. Ensure that the processes are documented in detail, covering scope, objectives, roles and responsibilities.

2. Examine the above-mentioned processes, procedures, and technical measures to confirm their compliance with the organization’s vulnerability management policy, as well as with relevant regulatory requirements and industry best practices.

3. Confirm that the above-mentioned processes, procedures, and technical measures are concretely and appropriately applied by involved parties in their day-to-day operations.

4. Inspect whether the above-mentioned processes, procedures, and technical measures are monitored against sets of efficacy and efficiency metrics / indicators.

5. Inspect whether the above-mentioned processes, procedures, and technical measures are periodically reviewed and updated by responsible parties.

6. Verify the MP uses SCA tools to manage vulnerabilities in the vast number of open-source libraries used in the ML ecosystem (e.g., numpy, pandas, scikit-learn).

7. Confirm a process is in place to test and validate library updates to ensure they don’t break model training or inference processes.

TVM-07: Penetration Testing

Control Specification

Define, implement and evaluate processes, procedures and technical measures for the periodic performance of penetration testing by independent third parties.

Auditing Guidelines for Model Providers (MP)

1. Verify that the MP has defined and documented processes, procedures, and technical measures for periodic penetration testing by independent third parties. Documentation must include scope, objectives, roles, and responsibilities.

2. Examine whether these processes comply with regulatory requirements and industry best practices.

3. Inspect whether these processes are aligned with the specific threat scenarios the MP faces—including risks specific to ML models.

4. Confirm that these processes are implemented and adhered to.

5. Verify that evidence from penetration testing activities is reviewed and translated into concrete remediation measures.

6. Inspect whether metrics and indicators are monitored to evaluate the efficacy and efficiency of the penetration testing program.

7. Inspect evidence that the processes are reviewed and updated at least annually or upon significant changes.

8. Verify that the MP conducts specialized penetration tests or AI Red Teaming focused on ML‑specific risks such as model inversion, extraction, and evasion attacks.

9. Review reports from these AI‑specific assessments and confirm that there is a documented process for mitigating identified vulnerabilities in models.

TVM-08: Vulnerability Remediation Schedule

Control Specification

Define, implement and evaluate processes, procedures and technical measures based on identified risks to support scheduled and emergency responses to vulnerability identification.

Auditing Guidelines for Model Providers (MP)

1. Verify that the Model Provider (MP) has defined processes, procedures, and technical measures to periodically (at least monthly) detect vulnerabilities on assets managed by the organization. Ensure that the processes are documented in detail, covering scope, objectives, roles and responsibilities.

2. Examine the above-mentioned processes, procedures, and technical measures to confirm their compliance with relevant regulatory requirements and industry best practices.

3. Confirm that the above-mentioned processes, procedures, and technical measures are concretely and appropriately implemented.

4. Inspect whether the above-mentioned processes, procedures, and technical measures are monitored against sets of efficacy and efficiency metrics / indicators.

5. Inspect whether the above-mentioned processes, procedures, and technical measures are periodically reviewed and updated by responsible parties.

TVM-09: Vulnerability Prioritization

Control Specification

Use a risk-based method for effective prioritization of vulnerability remediation using an industry recognized framework.

Auditing Guidelines for Model Providers (MP)

1. Verify that the Model Provider (MP) systematically adopts a method to support efforts in effectively and efficiently prioritizing remediations to vulnerabilities identified within the security perimeter.

2. Examine the above-mentioned method to verify that it adopts of a risk-based approach.

3. Examine the above-mentioned method to verify its compliance with industry recognized standards and frameworks.

TVM-10: Threat Response

Control Specification

Use a risk-based method for the prioritization and mitigation of threats, leveraging an industry-recognized framework to guide threat decision-making and protection measures.

Auditing Guidelines for Model Providers (MP)

1. Verify threat modeling on training pipeline (e.g., poisoning, data leakage).

2. Assess use of red-teaming/penetration testing on model outputs.

3. Validate risk scoring of model-related threats (e.g., by attack impact on model fidelity).

4. Ensure mitigation actions are guided by severity and exposure risk.

5. Check use of privacy-enhancing techniques (e.g., DP, encryption, federated learning) based on threat severity.

6. Review documentation of past threat incidents and response actions.

TVM-11: Vulnerability Management Reporting

Control Specification

Define and implement a process for tracking and reporting vulnerability identification and remediation activities that includes stakeholder notification.

Auditing Guidelines for Model Providers (MP)

1. Verify that the Model Provider (MP) has defined a process to systematically document both the vulnerabilities identified within the security perimeter and the activities implemented to remediate them. Ensure that the process is documented in detail, covering scope, objectives, roles and responsibilities.

2. Examine the above-mentioned process to verify that it includes a notification phase to relevant stakeholders.

3. Confirm that the above-mentioned process is communicated and thoroughly comprehended by relevant parties.

4. Confirm that the above-mentioned process is concretely and appropriately implemented by responsible parties.

5. Inspect whether the above-mentioned process is monitored against sets of efficacy and efficiency metrics / indicators.

6. Inspect whether the above-mentioned process is periodically reviewed and updated by responsible parties.

TVM-12: Vulnerability Management Metrics

Control Specification

Establish, monitor and report metrics for vulnerability identification and remediation at defined intervals.

Auditing Guidelines for Model Providers (MP)

1. Verify that the Model Provider (MP) has defined metrics and indicators for vulnerability identification and remediation at defined intervals.

2. Inspect whether the above-mentioned metrics and indicators are concretely and continuously monitored.

3. Inspect whether the above-mentioned metrics and indicators are periodically reviewed and updated by responsible parties.

4. Inspect whether the evidence emerged during the monitoring of the above-mentioned metrics and indicators is documented in appropriate executive and technical reports.

5. Inspect whether the above-mentioned reports are timely shared and actively discussed with all relevant parties to support decision making.

TVM-13: Guardrails

Control Specification

Define and implement processes, procedures and technical measures to apply guardrails to the AI system. Continuously evaluate guardrails for changes in regulatory requirements and risk scenarios.

Auditing Guidelines for Model Providers (MP)

1. Verify that the Model Provider (MP) has defined processes, procedures, and technical measures to apply guardrails to the AI system. Ensure that the processes are documented in detail, covering scope, objectives, roles and responsibilities.

2. Examine whether the above-mentioned processes, procedures, and technical measures are compliant with relevant regulatory requirements and industry best practices.

3. Confirm that the above-mentioned processes, procedures, and technical measures are concretely and appropriately implemented.

4. Inspect whether the above-mentioned processes, procedures, and technical measures are monitored against sets of efficacy and efficiency metrics / indicators.

5. Inspect whether the above-mentioned processes, procedures, and technical measures are periodically reviewed and updated by responsible parties.

UEM: Universal Endpoint Management

UEM-01: Endpoint Devices Policy and Procedures

Control Specification

Establish, document, approve, communicate, apply, evaluate and maintain policies and procedures for all endpoints. Review and update the policies and procedures at least annually, or upon significant changes.

Auditing Guidelines for Model Providers (MP)

1. Verify that the MP has established and documented endpoint device management policies and procedures, covering scope, objectives, roles, responsibilities, and relevant contexts (e.g., RandD vs. production).

2. Inspect whether these policies are compliant with applicable regulatory requirements, industry best practices, and specific risks to the MP (e.g., GPU servers).

3. Verify formal approval by authorized leadership.

4. Ensure the policies have been communicated effectively to all stakeholders and understood.

5. Confirm they are appropriately applied in day‑to‑day operations by involved parties.

6. Verify monitoring of effectiveness through metrics and indicators.

7. Inspect evidence of periodic review and update of policies (at least annually or after significant system changes).

8. Verify that the endpoint policy explicitly covers all RandD workstations and servers used for model training and inference.

9. Confirm the policy addresses secure configuration and hardening of high‑powered GPU servers.

UEM-02: Application and Service Approval

Control Specification

Define, document, apply and evaluate a list of approved services, applications and sources of applications (stores) acceptable for use by endpoints when accessing or storing organization-managed data.

Auditing Guidelines for Model Providers (MP)

1. Verify that the Model Provider (MP) has defined processes, procedures, and technical measures to identify and implement updates for applications that used by the endpoints involved in model development, training testing or inference, in order to mitigate risks of accessing unapproved services and it’s sources . Ensure that the processes are documented in detail, covering scope, objectives, roles and responsibilities.

2. Examine the above-mentioned processes, procedures, and technical measures to confirm their compliance with the organization’s relevant regulatory requirements and industry best practices.

3. Confirm that the above-mentioned processes, procedures, and technical measures are concretely and appropriately applied by involved parties in their day-to-day operations.

4. Inspect whether the above-mentioned processes, procedures, and technical measures are monitored against sets of efficacy and efficiency metrics / indicators.

5. Inspect whether the above-mentioned processes, procedures, and technical measures are periodically reviewed and updated by responsible parties.

UEM-03: Compatibility

Control Specification

Define and implement a process for the validation of the endpoint device’s compatibility with operating systems and applications.

Auditing Guidelines for Model Providers (MP)

1. Verify the existence of a formally approved Device Compatibility Policy, covering OS and hardware platforms (e.g., x86, ARM, GPU clusters) relevant to AI model development, testing, and deployment.

2. Confirm that the policy includes minimum hardware requirements, testing procedures, and secure configuration standards for AI workloads, especially in resource-intensive environments.

3. Inspect whether the policy mandates documentation and release notes, with compatibility matrices, supported versions, known limitations, and change tracking.

4. Validate implementation through operational artifacts, including QA records, performance benchmarks, compatibility logs, release notes, and periodic audit evidence.

5. Ensure compatibility procedures support model accuracy and security objectives, including graceful handling of resource constraints and alignment with applicable standards or regulations.

UEM-04: Endpoint Inventory

Control Specification

Maintain an inventory of all endpoints used to store, access and process company data.

Auditing Guidelines for Model Providers (MP)

1. Verify that the Model Provider has a formally documented Endpoint Inventory Policy, approved under governance, covering key aspects such as scope, roles, and responsibilities.

2. Confirm the policy defines criteria for categorizing endpoints used in AI model development and deployment, specifies update intervals and triggers, mandates collection of key attributes like OS, AI frameworks, and device location, and includes regulatory considerations.

3. Ensure the policy requires periodic audits to reconcile the inventory with physical assets and specifies secure removal of devices storing AI data.

4. Validate implementation by reviewing inventory dashboards, device tagging records, reconciliation evidence, decommissioning logs, and integration with AI workflows or asset management systems.

5. Confirm that real-time or near-real-time inventory visibility is maintained through automated tools, and that compliance with applicable data protection regulations is demonstrated.

UEM-05: Endpoint Management

Control Specification

Define, implement and evaluate processes, procedures and technical measures to enforce policies and controls for all endpoints permitted to access systems and/or store, transmit, or process organizational data.

Auditing Guidelines for Model Providers (MP)

1. Verify that the Model Provider has a documented Endpoint Management Policy, approved under formal governance, covering endpoint lifecycle management for AI model environments.

2. Inspect whether the policy defines standardized configurations, patching of OS and AI frameworks, centralized device management, and secure decommissioning procedures.

3. Confirm the policy addresses applicable data protection regulations, incident response integration, and periodic compliance reviews for AI-specific endpoints.

4. Review implementation evidence such as endpoint configuration records, patch and update logs, monitoring dashboards, incident records, and decommissioning logs.

5. Verify the use of centralized management tools integrated with AI security controls to maintain compliance, optimize performance, and enforce security measures across endpoints.

UEM-06: Automatic Lock Screen

Control Specification

Configure all relevant interactive-use endpoints to require an automatic lock screen.

Auditing Guidelines for Model Providers (MP)

1. Verify that the Model Provider has a documented Automatic Lock Screen Policy, with formal governance approval and clear definitions of scope, responsibilities, and applicability.

2. Inspect whether the policy defines inactivity timeouts tailored to AI data sensitivity, specifies secure unlock methods, handles exceptions for AI workloads, and aligns with data protection regulations.

3. Confirm that the policy applies to all relevant device types and OS platforms, supports uninterrupted AI workloads, and mandates periodic lock screen audits.

4. Review implementation evidence such as device configurations, compliance reports, exception records, audit results, and impact testing on AI processes.

5. Verify that lock screen enforcement is automated through endpoint management tools integrated with AI workflow monitoring and security controls.

UEM-07: Operating Systems

Control Specification

Manage changes to endpoint operating systems, patch levels, and/or applications through the company’s change management processes.

Auditing Guidelines for Model Providers (MP)

1. Verify that the MP has a formal Change Management Policy for AI‑processing endpoints (workstations, GPU servers, edge devices), specifying OS selection criteria, patch schedules, and AI‑workload requirements.

2. Inspect the policy to ensure it includes secure baseline configurations (service removal, secure boot), roles/approval processes, and scheduled policy reviews.

3. Confirm the policy mandates pre‑deployment testing of OS patches and driver/framework updates for AI training/inference stability.

4. Verify that the policy requires ongoing vulnerability scanning of OS components on AI endpoints and references relevant compliance or data‑protection standards.

5. Review operational artifacts (OS inventories, patch/update logs, QA test results, vulnerability remediation records, and audit reports) to validate adherence.

UEM-08: Storage Encryption

Control Specification

Protect information from unauthorized disclosure on managed endpoint devices with storage encryption.

Auditing Guidelines for Model Providers (MP)

1. Verify that the MP has a formally approved Storage Encryption Policy defining scope, governance, roles, and review intervals.

2. Inspect the policy to confirm it mandates strong encryption standards for AI data at rest, key management procedures, exemptions criteria, and compliance with industry regulations.

3. Confirm the policy requires encryption throughout the model lifecycle (on workstations, GPU servers, edge devices) and enforces secure key provisioning, rotation, and retirement.

4. Verify that the policy includes automated monitoring of device encryption status, key‑rotation audits, and integration with corporate IT full‑disk encryption requirements.

5. Review artifacts (device encryption inventories, key rotation and recovery documentation, configuration baselines, audit logs, and incident‑response plans) to ensure adherence.

UEM-09: Anti-Malware Detection and Prevention

Control Specification

Configure managed endpoints with anti-malware detection and prevention technology and services.

Auditing Guidelines for Model Providers (MP)

1. Verify that the MP has a documented Anti‑Malware Policy, formally approved, defining scope, governance, roles, and review intervals.

2. Inspect the policy to confirm it mandates reputable endpoint protection solutions, secure default configurations, timely signature/engine updates, and advanced detection (heuristic or AI‑driven).

3. Confirm the policy specifies incident‑response steps for AI workload endpoints, including isolation procedures and model‑integrity checks after remediation.

4. Verify the policy applies to all AI‑processing devices such as RandD workstations, GPU servers, edge/IoT, and requires integration with central monitoring and key management.

5. Review artifacts (installation logs, configuration records, compliance dashboards, update logs, incident‑response documentation, advanced detection reports, and periodic audit results) to ensure adherence.

UEM-10: Software Firewall

Control Specification

Configure managed endpoints with properly configured software firewalls.

Auditing Guidelines for Model Providers (MP)

1. Verify that the MP has a documented Software Firewall Policy, approved by leadership, covering developer workstations, GPU servers, IoT/edge devices, and storage nodes.

2. Inspect the policy to confirm it mandates default‑deny firewall rules with selective allow‑lists for AI data‑flow ports and specifies logging requirements.

3. Confirm the policy includes formal exception processes for AI workflows and patch/update procedures for firewall software.

4. Verify that the policy enforces centralized management or endpoint agents to deploy and monitor firewall configurations uniformly.

5. Review artifacts (rule‑set templates, deployment dashboards, change‑approval records, log‑analysis reports, and periodic penetration‑test results) to ensure adherence.

UEM-11: Data Loss Prevention

Control Specification

Configure managed endpoints with Data Loss Prevention (DLP) technologies and rules in accordance with a risk assessment.

Auditing Guidelines for Model Providers (MP)

1. Verify that the MP has a documented DLP Policy, formally approved, defining scope, governance, roles, and review intervals.

2. Inspect the policy to confirm it mandates classification‑based rules protecting AI model artifacts, training data, and inference outputs, with integration into data pipelines and model registries.

3. Confirm the policy requires DLP controls on devices handling AI assets like workstations, GPU servers, edge, and defines response steps (quarantine, forensic analysis, model‑integrity checks).

4. Verify that the policy specifies exception handling for legitimate large exports, alignment with data‑protection regulations, and periodic effectiveness testing.

5. Review artifacts (data classification documentation, endpoint DLP settings, monitoring dashboards, incident records, exception logs, and audit reports) to ensure adherence.

UEM-12: Remote Locate

Control Specification

Enable remote geo-location capabilities for all managed mobile endpoints, according to all applicable laws and regulations.

Auditing Guidelines for Model Providers (MP)

1. Verify that the Model Provider has a documented Remote Locate Policy, approved under formal governance, covering scope, responsibilities, and operational control.

2. Inspect whether the policy defines who can request location tracking, mandates privacy protections, addresses AI-driven automated tracking, sets secure storage and deletion timelines for location data, and integrates with incident response.

3. Confirm the policy includes periodic audits of locate activity and ensures strong security controls for location tracking on AI-specific devices.

4. Review implementation evidence such as authorization records, privacy documentation, technical integration details, locate logs, and incident response documentation.

5. Verify that remote locate actions are restricted to authorized personnel using encrypted, authenticated platforms to prevent interception or unauthorized access.

UEM-13: Remote Wipe

Control Specification

Define, implement and evaluate processes, procedures and technical measures to enable the deletion of company data remotely on managed endpoint devices.

Auditing Guidelines for Model Providers (MP)

1. Verify that the Model Provider has a documented Remote Wipe Policy, approved under formal governance, defining scope, roles, and responsibilities.

2. Inspect whether the policy sets strict approval criteria for remote wipes, aligns actions with AI model classifications, addresses relevant data protection laws, distinguishes full vs. partial wipes, integrates with incident response, and mandates regular testing.

3. Confirm the policy includes specialized procedures for AI hardware and environments, ensuring wipe operations protect AI model integrity while preserving essential system functions.

4. Review implementation evidence such as authorization records, technical execution logs, compliance reports, and incident response documentation.

5. Verify that remote wipe commands are executed through secure, authenticated tools with role-based restrictions, and that the policy is regularly tested for effectiveness.

UEM-14: Third-Party Endpoint Security Posture

Control Specification

Define, implement and evaluate processes, procedures and technical and/or contractual measures to maintain proper security of third-party endpoints with access to organizational assets.

Auditing Guidelines for Model Providers (MP)

1. Verify that the Model Provider has a documented Third-Party Endpoint Security Agreement, approved under formal governance, defining scope, roles, and responsibilities.

2. Inspect whether the agreement requires security due diligence for vendor endpoints handling AI models or data, mandates security clauses in contracts, enforces continuous security monitoring, requires prompt incident notification, defines secure model/data disposal procedures, references regulations, and includes periodic audits.

3. Confirm that the agreement covers all stages of the AI lifecycle and that vendors implement advanced security controls for AI workloads and development environments.

4. Review implementation evidence such as risk assessments, contract security clauses, compliance dashboards, incident reports, and forensic analysis records.

5. Verify that vendors follow security best practices for AI-related endpoints by assessing audit reports, monitoring records, model protection procedures, and incident management evidence.