Part of Future Forward Initiatives
Project kickoff · June 2026
Turning existential AI concern into auditable controls — an expert-led extension of the AI Controls Matrix, validated through real pilot audits.
Underwriting independent, non-commercial research on the gravest risks of transformational AI — in the public interest.
As AI systems approach transformational capability, the risks stop being purely theoretical. The Catastrophic Risk Annex convenes AI safety, cybersecurity, and national-security professionals to define a concrete set of catastrophic-AI controls — extending the AI Controls Matrix — and to prove those controls work through pilot audits with real organizations.
Author a rigorous control set extending the AICM, targeting the failure modes that could cause catastrophic or existential harm.
An interdisciplinary group of AI safety, cybersecurity, and national-security professionals authors and stress-tests the controls.
Prove the controls are auditable and meaningful by running pilot audits against real AI systems and organizations.
Expert Group convened; scope and control taxonomy defined.
Draft the catastrophic-AI control set as an AICM extension.
Validate controls against real systems; refine on findings.
Published, validated Catastrophic Risk Annex.
Catastrophic risk sits at the intersection of fields that rarely share a framework. The Annex deliberately brings them together.
AI Safety
Cybersecurity
National Security
An interdisciplinary Expert Group is convened and at work authoring and stress-testing the controls, with the AI Resilience Center of Excellence as a stakeholder. Membership is open to qualified experts and CSAI Foundation Executive Advisory Committee members.
Executive Advisory Committee members and qualified experts can join the Expert Group; organizations can express interest in a pilot audit.