Can GenAI Services Be Trusted? | At the Discovery of STAR for AI
Published 01/29/2025
Whenever new technologies are introduced into our personal lives, organizations, or even society as a whole, we always ask the same question: Can I trust it?
Most recently, how many of us have asked ourselves whether, how, and within which limits we can trust LLM/GenAI services and their providers? This is a legitimate question, given that we are faced with an innovation that holds the potential to transform our lives profoundly. Policymakers and regulators seem to be grappling with similar concerns. Meanwhile, GenAI service providers are asking: How can we earn the trust of our customers, policymakers, regulators, and markets?
The same question was posed during the early days of cloud computing: Can I trust the cloud? Back then, the Cloud Security Alliance (CSA) brought together the expertise of providers, customers, auditors, and regulators to address the complex matters of trust and assurance in cloud computing, and we created the STAR Program. Building on that experience, CSA has introduced STAR for AI—a pioneering initiative inspired by those early days of cloud computing.
I’m Daniele Catteddu, CTO at CSA and Co-Founder of the STAR Program. In this blog, I will discuss current GenAI services governance, trust, risk management, and compliance challenges and will introduce the STAR for AI initiative that aims to establish a standard for GenAI services assurance.
What does it mean for a GenAI service provider to be trustworthy?
Or even better, what does it mean for a GenAI service to be trustworthy?
For our purpose, we’ll define a Trustworthy GenAI Service as one that is committed to serving humanity responsibly and ethically via safe technologies.
It is offered by a system that is robust, reliable, resilient, explainable, controllable, transparent, accountable, protects privacy, fosters fairness, and complies with all applicable laws and regulations.
Back in the early days of cloud computing, the challenges were:
- The introduction of a new business and technological paradigm (someone else’s computer syndrome)
- Confusion on how to govern data (where is my data in the cloud obsession)
- Confusion on how to govern new categories of services (SaaS Governance nightmare)
- Confusion about how to use and integrate the new technology and services into existing platforms and whether to revolutionize existing habits, systems, and platforms based on the new technology
- Confusion in the legal and regulatory landscape
- Lack of standards (both technical and assurance/quality standards) and need to retrofit existing ones
For AI, the situation appears to be similar in many ways, with a few notable differences. The paradigm shift does not only affect businesses and technologies; it goes deeper into the very fabric of our society and personal lives. The new syndrome is not someone else’s computer but someone else’s (else’s) brain. The risk of over-reliance on the AI Oracle did not exist with the cloud.
We have the same confusion regarding data governance (the obsession with “who is using my data for training?”) and service governance (How to prevent errors, abuses, and unwanted uses? How to understand what is real and what is fake? What is human vs. AI-generated? Shadow-AI paranoia, Evil-GTP fear, soon-to-be Evil AI Agent fear).
Similarly, many organizations face uncertainty, ranging between full embracement, timid acceptance, refusal, and complete denial.
Should I stay or should I go? The same old-school human resistance to change and legacy problem, exacerbated by the much stronger disruptive potential of GenAI compared to the cloud and mitigated by the substantially more advanced capabilities offered by GenAI.
What’s the current state of AI security assurance and trust?
The political, legal and regulatory landscape is tough. The geo-strategic interests at stake are at a scale humanity has seen only during the nuclear race during WW2. Acknowledging GenAI as a potential existential threat underscores the urgency of robust governance. However, I recognize that framing it in terms of the nuclear race may hinder productive dialogue by evoking a zero-sum, adversarial mindset. Instead, GenAI's risks should be addressed through a combination of cooperative international agreements, transparent development practices, and enforceable safety standards.
We all know we are playing with fire - everyone is trapped in this prisoner’s dilemma. Nobody is willing to stop the race to Superintelligence. Some believe that regulations are premature and potentially harmful. Others believe that in the absence of them, we’ll quickly lose control of society as we know it and go down the path of a dystopian future.
The EU, China, and Brazil have already adopted regulations: the EU AI Act, Generative AI Regulations (e.g. Measures for the Administration of Generative Artificial Intelligence Services) and Brazilian AI Bill, respectively. The EU AI Act, which I know a bit better, aims to strike a good balance between innovation, safety, and accountability. At the same time, the USA government appears to espouse a self-regulatory approach. In between, there are several other countries in the process of figuring out the principles and laws to rule and guide the transition toward a new AI-driven society.
In the background, there is also a battle between parts of the scientific community. One side has the urge to slow down, or even pause, the exponential progress of AI until we have more solid solutions to the alignment problem. Others firmly believe we need to immediately commercialize these new technologies to build an AI-aware society and favor the embracement of the positive effect of GenAI.
The standards are clearly in their infancy. We are slowly building the vocabularies and taxonomies to understand each other. However, we can anticipate that the consensus-building exercise that stands behind the standard creation seems to clash with the light-speed progress of AI technologies and the continuous release of solutions.
Someone could summarize the situation as a bit chaotic...
What is CSA going to do about it?
In this chaos, at CSA, we decided to use the same principles, tips, and tricks that proved useful for cloud computing assurance to build an approach that can offer a solid, robust, understandable, and measurable “something” —a starting point— to reiterate and improve continuously. Over time, CSA hopes to achieve a better approximation of the measure of trustworthiness of the AI services that we’ll consume.
This little “something” we are building is called STAR for AI. For those not familiar with CSA, STAR is the acronym for Security, Trust, Assurance, and Risk. STAR is CSA’s program for cybersecurity assurance, governance, and compliance, and its initial focus and scope was cloud computing.
In the cloud computing industry, the STAR program is widely used as an indicator of cloud service trustworthiness, both in the private and public sector domains. For instance, some countries like Italy officially use it as a mechanism of adherence to national requirements for the public sector and critical infrastructures.
And we hope to achieve the same success with our latest initiative.
STAR for AI will focus on Multimodal GenAI services and consider their overall value chain. This means the evaluation will have within its scope one or more of the following components:
1) Cloud / Processing Infra / GenAI Operations, 2) Models, 3) Orchestrated Services, 4) Applications, and 5) Data (for reference please read the CSA LLM Threat Taxonomy).
Additionally, the program will evaluate the security of AI service usage.
Our goal is to create a trustworthy framework for key stakeholders to demonstrate safety and security, whether they are a Gen-AI frontier model owner, (added-value) AI service developers and providers, or AI services business users.
The program will focus on technical and governance aspects related to cybersecurity. Additionally, it will cover aspects of safety, privacy, transparency, accountability, and explainability as far as they relate to cybersecurity.
The audit and evaluation approach will be risk-based, i.e., the suitability of the controls in place will be established based on the risks to which the service is exposed. The risk-based approach will ensure the audit and evaluation process is relevant to the context and use case.
The program will leverage existing auditing and accreditation standards, which might be AI-specific (e.g., ISO/IEC 42001-2023) or more general (ISO27001, ISO17021, 17065, etc.). It will include both point-in-time and continuous auditing.
Introducing the AI Controls Matrix
As a first step, we are establishing a control framework for GenAI service security. The framework is open, expert-driven, consensus-based, and vendor-agnostic. Its ambition is to repeat and improve on the success of the Cloud Controls Matrix and become an industry de facto standard.
The framework is called the CSA AI Controls Matrix (AICM).
We released it for peer review just before the end of 2024, hoping to offer our community a present for the beginning of the new year.
The initial version of AICM was designed to follow common sense principles of trustworthiness. We defined a taxonomy, created threat scenarios, and identified control objectives to mitigate the threats. The AI control objectives were framed in the CSA Cloud Control Matrix’s template and structure. We leveraged the strengths of the CCM model, and customized and improved where necessary. The team of volunteers contributing includes experts from industry, academia, and governmental bodies. Needless to say we also used the support of GenAI tools.
The current version of the AICM is structured in 18 domains. 17 are in common with the CCM, plus the addition of the Model Security Domain. There are 242 control objectives (37 of them AI-specific, 183 relevant to both AI and cloud, and 22 cloud-specific).
This is our first call for action. If you care about AI service trustworthiness and/or are a subject expert on the matter, you should review the current draft of the AICM and contribute to improving it. It might seem like a hyperbole, but having your name on the standard that will design the boundaries of GenAI service trustworthiness, means leaving a legacy in history.
Determining assessment mechanisms
The other foundational component of STAR for AI is the auditing scheme, the conformity assessment mechanisms that will be used to evaluate the adherence of a given GenAI service to the AICM requirements.
With the support of the auditing and assurance community, in 2024 we started to reason about the mechanisms fit for GenAI service cyber assurance evaluation. More in general, we started a discussion on the impact of AI on auditing and the future of assurance and trustworthiness.
We are exploring options. Several possible approaches are considered for use. Some are already existing and standardized (self-assessment, third-party audit, etc.), others are under development (continuous controls monitoring/auditing), and others might be introduced as a result of new technologies or new technical needs (e.g., Gen AI Auditing/Assessment).
Here comes our second call for action, and once again, it involves you taking a step forward, being a thought leader, and contributing to shaping the future of cyber assurance. If you would like to be involved in the creation of the auditing scheme, please get in touch.
An urgent call to action
Why are we building STAR for AI? Simple: Within our community, there’s a background voice that is increasing in volume by the second. It quickly became a scream. What we hear is a request to support the controlled adoption of GenAI services and the governance of its cybersecurity and safety. We need a mechanism to measure trustworthiness, and CSA is strategically committed to delivering such a solution.
To conclude, there is growing urgency in the market to establish reliable assurance and compliance mechanisms for governing GenAI services. This challenge is particularly complex as it intersects with broader ethical considerations and complex technology.
We face a notable paradox: even as we work to define parameters and metrics for GenAI trustworthiness, these technologies are already embedded in our organizations and personal lives. Adoption is only accelerating as organizations recognize the opportunities GenAI creates. Moreover, we are increasingly relying on GenAI tools for assessment and auditing processes, including autonomous decision-making. This creates the potential situation where we might depend on a technology to evaluate its own trustworthiness before we have established reliable methods to measure the integrity of the decisions the technology may take without human intervention.
While this situation doesn't call for panic, it does demand urgent attention. I encourage all of you to contribute your expertise to the STAR for AI initiative to help address these critical challenges.Related Articles:
Implementing CCM: Put Together a Business Continuity Management Plan
Published: 02/14/2025
7 Steps to Get Started with Security and Privacy Engineering
Published: 02/14/2025
How Repsol’s DLP Strategy Enables a Fearless Embrace of GenAI
Published: 02/13/2025