Continuous Auditing and Continuous Certification

By Alain Pannetrat, Senior Researcher at Cloud Security Alliance and Founder of Omzlo.com

For some cloud customers in sensitive or highly-regulated industries, such as banking or healthcare, “traditional” annual or bi-annual audits do not provide enough assurance to move to the cloud. To address the concerns of this segment of the industry, the Cloud Security Alliance is building STAR Continuous: an innovative framework designed to provide assurance to customers on a monthly, daily or even hourly basis.

The foundation of STAR Continuous is continuous auditing: the continuous evaluation of certain characteristics of an information system, mostly by automated means, in order to get near real-time assurance. Continuous audits can be used as a basis for a novel type of certification (or attestation) as well as for self-assessments. In many ways, the industry is already doing continuous auditing. Yet cloud customers cannot fully take advantage of it, due to lack of relevant standards and best practices.

Read on to learn more about the genesis and purpose of STAR Continuous.

When a certification or an attestation is not good enough

Certification and attestation schemes such as those offered by the CSA Open Certification Framework (OCF), ISO/IEC, or AICPA, have strongly contributed to the success of the cloud by providing many cloud customers the necessary assurance that the cloud service they are using meet relevant security requirements. These schemes rely on annual or biannual audits conducted by trusted independent auditors. However, for some cloud customers in sensitive or highly-regulated industries, such as banking or healthcare, the time elapsed between annual or bi-annual third-party audits is perceived as a “blind spot”: a much more frequent level of scrutiny is required.

Over the years, CSA has participated in several research initiatives with industry, public bodies and academia in order to develop new certification tools providing a more continuous level of assurance. Recently, as part of the European Commission-funded project EU-SEC, CSA participated in a pilot for the continuous certification of a cloud service for a major Spanish financial institution (LaCaixa) and successfully demonstrated the feasibility of providing continuous assurance to demanding cloud customers.

The continuous certification scheme CSA has developed extends a “traditional” certification scheme with a continuous process of automated checks. The whole process can be summarised in two consecutive phases: an initialisation phase and a continuous audit phase.

Initialisation phase:

The CSP undergoes a traditional third-party audit in order to obtain a certification or attestation. In addition, the CSP defines:

• A continuous certification target which comprises a set of security objectives, each associated with a policy that defines the assessment frequency (e.g. check every 4 hours).
• A set of tools capable of verifying that the security objectives are fulfilled..

The third party auditor involved in the certification checks:

• That the defined continuous certification target covers a satisfactory scope of the certified information system.
• That the reporting tools are trustworthy and fit-for-purpose.
• If this process is successful the continuous certification target is transmitted to the certification authority (i.e. CSA), which creates a corresponding entry for the cloud service in a dedicated public registry of continuously certified cloud services.

Continuous audit phase:

The third-party auditor periodically performs checks to confirm that the assessment tools are trustworthy (e.g. integrity checks).

The assessment tools continuously reports back to the certification authority (i.e. CSA) through a dedicated API the results of the assessment of each defined security objective, according to the frequency defined in policies within the continuous certification target:

• If a CSP reports in due time that all security objectives are met, the cloud service is marked as “compliant” in the corresponding entry in the public registry.
• If a CSP reports non-compliances or if the CSP fails to report about security objectives in due time, the entry will ultimately be removed from the public registry if the situation is not resolved with a predefined period of time.

It’s important to note that the public registry (STAR) will not provide details of non-compliances in order not to potentially compromise the security cloud services under scrutiny.

CSA’s research has highlighted that one of the biggest challenge in the process outlined above is the definition of the continuous certification target, and in particular the set security objectives that are used to assess an information system.

Let’s see why.

Security Level Objectives and Security Qualitative Objectives

Traditional certification typically relies on control frameworks such as the CSA Cloud Control Matrix or ISO/IEC 27002. These frameworks contain high-level control objectives that are interpreted by humans and translated into applicable technical or organisational security controls. This process is slow and complex and cannot be conducted on a daily or hourly basis. On the other hand, at least some of the applicable technical or organisational security controls can be evaluated automatically and frequently, if we are able to express them as quantifiable or qualifiable attributes of an information system, associated to metrics and expected results.

Thinking in terms of quantifiable or qualifiable attributes, metrics and expected results is, in fact, a familiar concept in the cloud, as embodied through Service Level Agreements (SLA), where cloud providers express expected results usually related to performance attributes of a cloud service, along with the metrics used to assess them. What has been done for performance in SLAs can also be done for security and the standardisation community has been working to build Security Level Agreements for cloud computing through the development of ISO/IEC 19086.

The continuous certification scheme CSA has developed uses ISO/IEC 19086 as a foundation, using its well-defined terminology and conceptual model. The standard notably defines 3 important concepts:

1. Metric: a standard of measurement that defines the conditions and the rules for performing the measurement and for understanding the results of a measurement.
2. Cloud service level objective (SLO): commitment a cloud service provider (ISO/IEC 17788:2014, 3.2.15) makes for a specific, quantitative characteristic of a cloud service (ISO/IEC 17788:2014, 3.2.8), where the value follows the interval scale or ratio scale.
3. Cloud service qualitative objective (SQO): commitment a cloud service provider (ISO/IEC 17788:2014, 3.2.15) makes for a specific, qualitative characteristic of a cloud service (ISO/IEC 17788:2014, 3.2.8), where the value follows the nominal scale or ordinal scale.

Consider for example, as a control objective, the need to define and regularly test business continuity plans. At a high level, such a control objective is difficult to quantify or measure explicitly, with a corresponding expected result. At a lower level however, we can identify many useful technical attributes of an information system that can be used to highlight the strength of business continuity plans. For instance, the number of successful backup restoration simulated per month/week, the recovery point actual, or data durability. Each one of these attributes can be tested and measured according to a metric, and corresponding objectives can be set. Moreover, these attributes can be tested automatically and regularly.

It turns out that this work of translating high-level control objectives into SLOs and SQOs is hard, due to the absence of existing guidance in the field. Just like we did for traditional certification through the creation of control frameworks, we now need to create standards for security attributes, metrics, SLOs and SQOs in order to enable the practical deployment of continuous audit-based certification.

It’s also a tool for self-assessment

The usefulness of a continuous auditing framework is clearly not limited to third-party certification for customers in sensitive industries. In fact, such a framework could be just as important and useful for organisations wishing to perform a continuous assessment of their cloud assets.

Again this will only reach its true potential if there is a standard set of security attributes, metrics, SLOs and SQOs that the industry adopts as a reference for continuous auditing, giving practitioners a meaningful reference to assess and relate the security of competing cloud services.

With the right platform, we can well imagine a continuous audit-based self-assessment that mirrors what the CSA CAIQ is doing today as a point-in-time assurance tool.

Continuous is already there

One major IaaS provider recently joked with us that there is never a day in the year where there is not at least one external auditor setting a foot in their data centres.

In order to do business today, cloud providers are obliged to be compliant with dozens of compliance schemes, both international and regional, or sector specific, such as ISO 27001, AICPA SOC, CSA STAR, PCI DSS, FedRamp, FISMA, HIPAA, or BSI C5 just to name a few. There is a lot of overlap in security requirements between these various assurance schemes. As a result, cloud service providers are under “continuous” scrutiny.

Moreover, as a natural part of information security management, most cloud providers and customers are using security tools that continuously assess the security of their information systems. Cloud security tool vendors have developed a rich set of data points and assessment mechanisms to address industry requirements. In many ways, what we call SQOs, SLOs and metrics, already exist, albeit under different names.

Unfortunately many of these efforts remain invisible to cloud customers, due to the lack of supporting standards and best practices.

What Cloud Security Alliance is doing

By creating STAR Continuous, the Cloud Security Alliance aims to build the next generation of certification and self-assessment tools, based on a continuous auditing.

In this process, we established the following goals:

• Capitalise on existing standards, such as ISO/IEC 19086, avoid reinventing the wheel.
• Be technological neutral: continuous auditing tools should be freely selectable by the industry, as long as they can demonstrate that they are trustworthy and fit-for-purpose.
• Strike a balance between transparency and security, while providing continuous assurance to all cloud customers.
• Complement but not replace traditional certification.

In the context of this effort, the Cloud Security Alliance is launching a new initiative dedicated to the definition of security attributes and metrics associated with the control objectives defined within our Cloud Control Matrix (CCM), the CSA Continuous Audit Metrics Working Group.

We are now seeking the help of cloud customers, cloud providers, security tool vendors, auditors and all relevant experts in order to define the very first industry-wide catalogue of security attributes and metrics for continuous auditing.

Please get in touch with us to participate in this exciting project! (email to: [email protected])