RiskRubric Updates: AI Risk Assessment for the Agentic Era

RiskRubric, CSA’s evidence-based risk rating system for AI technologies, is getting some timely updates. These updates aim to expand AI risk assessment beyond the model layer, reduce blind spots, and address maturing threats.

The upcoming updates to RiskRuric include:

• A multi-scanner ecosystem powered by independent partners Deloitte Italy, PointGuard, and Tumeryk.
• Expanded assessment coverage beyond AI models to include MCP servers and AI agents.
• Modernized evaluation pillars addressing emerging autonomous AI risks.
• A new scoring model to provide greater transparency into assessment validation.

Today you can visit the new landing page, review the updated scoring model, and read the complete V2 Concept Paper. The new platform will launch in Q3 2026.

Below, learn more about these updates and how they further CSAI's efforts toward AI reliability and transparency.

 

RiskRubric and the CSAI Mission

CSA launched RiskRubric in September 2025. This original version introduced a structured framework for rating AI models on six dimensions of trustworthiness: Transparency, Reliability, Security, Privacy, Safety, and Reputation. RiskRubric scans each model with an evaluation engine that combines automated red-teaming with open-source intelligence collection. The indicators for each trustworthiness pillar combine into a formula that produces a 0-100 score and A-F grade.

Last year’s launch demonstrated that we can successfully express AI risk as a structured, evidence-based score. It also showed that the resulting grades are useful in procurement, deployment, and governance decisions.

Since then, we've integrated RiskRubric into CSAI’s AI Risk Observatory. The AI Risk Observatory aims to help the industry gain visibility into how agents behave, fail, and introduce risk.

We don’t want developers to have to default to whatever AI tool is most popular or powerful. RiskRubric provides them the visibility that the AI Risk Observatory is aiming to achieve. By simply checking a RiskRubric scorecard, developers can understand whether a given model meets their thresholds for trustworthiness. This gives developers confidence that they’re building on a secure foundation.

 

Time for an Update

In the last nine months, we've noticed some key changes and points of clarity:

• The boundary of what counts as “the AI system” has moved. Production deployments increasingly include not only models but also MCP servers and AI agents. Risk assessment confined to the model layer leaves substantial attack surface unmeasured.
• A single evaluation engine represents a single viewpoint, a single point of failure, and a constraint on growth. The maturing field needs cross-evaluator validation to reduce blind spots and scale assessment volume.
• The threat model has matured. Concerns once treated as advanced (goal hijacking in agents, autonomy escalation, etc.) are now mainstream. They require first-class treatment in any credible rating framework.

The updates to RiskRubric respond to these pressures while preserving the original version's core principles.

 

Expanding the Scope

Instead of just looking at a narrow definition of AI models, RiskRubric will now evaluate three distinct service types:

• AI Models: Trained inference engines accessed via API or model weights. Evaluated on their response behavior under adversarial and benign prompts.
• MCP Servers: MCP endpoints that expose tools, resources, and prompts for consumption by models or agents. Evaluated on tool-surface integrity, schema disclosure, tool-call abuse resistance, supply-chain risk, transitive trust handling, etc.
• AI Agents: Autonomous systems composed of one or more models, tools, memory, and a planning loop. Evaluated on end-to-end task execution.

Organizations are increasingly relying on interconnected systems that combine models, tools, memory, and autonomous decision-making rather than standalone models operating in isolation. As a result, meaningful risk assessment must account for the full system architecture and the interactions between its components. By extending coverage to MCP servers and AI agents, RiskRubric provides a more complete view of the risks organizations face when deploying modern AI solutions.

 

Establishing a Scanner Ecosystem

The original version of RiskRubric featured a single evaluation engine (called a “scanner”). RiskRubric will now support a scanner ecosystem. Various partner organizations will operate their own scanners that produce compliant scores for one or more service types. CSA will govern the methodology.

To qualify as a a RiskRubric scanner, an evaluation engine must:

• Implement the indicator set for at least one service type
• Retain raw test artifacts for audit
• Declare the version of the RiskRubric methodology that it implements
• Disclose when scores are for the products of its operating organization or an affiliated entity
• Produce stable scores across re-runs of the same target, within reasonable bounds

The scanner ecosystem represents a shift from a centralized evaluation model to a federated approach. No single evaluator can anticipate every failure mode, attack technique, or emerging risk. Enabling multiple independent organizations to generate evidence under a shared methodology results in a stronger and more durable framework. It benefits from methodological diversity without sacrificing consistency, transparency, or comparability.

 

Maturing the Scoring Model

The RiskRubric scoring model will still have six pillars, but we are replacing the “Reputation” pillar with “Excessive Agency.” The Reputation pillar conflated vendor track record with model behavior. On the other hand, the new Excessive Agency pillar examines whether the service stays within its scope and boundaries. This is a necessity as agents become ever more autonomous.

The weights will be adjusted as follows:

• Transparency: 15% → 16%
• Reliability: 20% → 16%
• Security: 25% → 20%
• Privacy: 20% → 16%
• Safety: 15% → 16%
• Reputation: 5% → 0%
• Excessive Agency 0% → 16%

Additionally, the scoring model will now include tuning based on the service type and a new confidence index based on the number of distinct scanners that contributed to the score.

These updates make RiskRubric scores more representative of real-world risk while increasing AI transparency. Different service types present different risk profiles, making a one-size-fits-all weighting model increasingly difficult to justify. The confidence index improves interpretability by helping users understand what a score is and how much evidence supports it. RiskRubric scores will be easier to trust, explain, and align with the realities of modern AI systems.

 

A Timely Partnership

Our independent scanner ecosystem kicks off with scanners by Deloitte Italy, PointGuard, and Tumeryk.

Risk assessment is strongest when it incorporates multiple perspectives. Different evaluators bring different testing methodologies, research priorities, and operational experiences. By allowing multiple organizations to contribute evidence under a common methodology, RiskRubric can increase confidence in assessment results.

This approach mirrors how mature assurance ecosystems operate. Security testing, financial auditing, and standards conformance assessments all benefit from independent validation rather than reliance on a single evaluator. A scanner ecosystem enables RiskRubric to move in the same direction, creating a more resilient and transparent framework.

These changes are arriving at a pivotal moment for the industry. Organizations have rapidly moved beyond standalone models to connected systems composed of models, tools, agents, memory, and external services. At the same time, regulators, enterprise buyers, and security teams are demanding greater visibility into how these systems behave.

The scale of this transition makes it increasingly difficult for any single organization to evaluate every service, every deployment pattern, and every emerging threat. A collaborative ecosystem allows RiskRubric to expand alongside the market while maintaining the transparency and rigor that made the original framework valuable.

 

Make Informed Decisions About LLMs

RiskRubric translates complex technical findings into accessible scores, grades, and supporting indicators. This helps developers, security teams, and executives make more informed decisions about the AI systems they use and trust.

Whether you're selecting a model, integrating an MCP server, or deploying an agent, RiskRubric provides an objective starting point. Rather than relying on vendor claims, marketing materials, or anecdotal reports, organizations can reference standardized assessments. We built these assessments on transparent methodologies and measurable evidence.

Today, you can explore the new RiskRubric page, review the updated scoring, and read the complete concept paper. The new platform and scanner ecosystem launch in Q3 2026. Join us as we build a more trustworthy ecosystem for AI risk assessment.