Meeting and Scaling Compliance with Intentional IaC Design

Originally published by Coalfire.

Written by Christian Stano, Senior Consultant, Cloud Services, Coalfire.

Key takeaways

• Utilizing consolidated, “smart” compliance-aware Terraform module design decreases audit risk and cost and maintains or accelerates developer velocity by shifting the burden of compliance away from application engineers.
• Reducing the sprawl of compliance codified with Terraform provides a single source of truth for compliance and policy reviews of infrastructure, increasing confidence in an enterprise’s audit readiness and ability to scale readiness to multiple business units.
• Consolidating automated compliance hardening with a “write once, use many times” design pattern unlocks compliance program scalability across the enterprise.

Compliance programs can scale with the right approach

Building infrastructure in compliance with frameworks such as GDPR, HIPAA, PCI, and FedRAMP™ creates new opportunities for enterprises to enter restricted markets and broaden the adoption and impact of their products. Too often, however, leaders are deterred from starting, completing, or growing in the product compliance journey due to the number of moving pieces in the build process, not to mention justifying the significant investment required. By nature, compliance audits are expensive from an overhead, infrastructure, preparation, and process perspective.

The cloud infrastructure supporting a business unit is a significant portion of the compliance equation. With the traditional DevOps model, each team becomes responsible for understanding and implementing compliant infrastructure, a nuanced process with cloud service providers. This process quickly leads to a sprawl of compliance shouldered primarily by siloed engineers, leading to inefficiencies and decreased confidence from human error. I’m here to share some successful Terraform practices, design patterns, and automation to help companies combat this challenge by streamlining and scaling compliant infrastructure.

Combatting compliance sprawl

As engineers with the cloud services team, we typically see that meeting specific compliance frameworks usually becomes a business case after an organization has defined and matured tens to hundreds of Terraform repositories and modules. In our experience, this Terraform sprawl is one of the root problems preventing companies from effectively meeting, auditing, and scaling Terraform to meet compliance controls.

Cracks begin to show in attempts to fork or lift and shift these existing infrastructure codebases and practices to meet compliance frameworks. The typical symptoms are:

• Compliance experts are spread thin, attempting to review and scale programs to meet stringent, granular controls in too many places with differing standards and syntaxes. The compliance experts experience burnout quickly.
• Engineers exhibit frustration or frequently hit blockers trying to make compliance decisions on module parameters. Modules require frequent rewrites, conflicting releases, and forks to allow for new inputs not previously planned for.

As a result, compliance sprawls across repos, modules, code snippets, and teams. The potential for human error with this sprawl skyrockets, contributing to potential audit failures, unforeseen costs, pushed timelines, developer burnout, and ultimately stymied compliance programs trying to scale.

Leveraging automation to solve compliance scalability issues

Compliance can scale, and it does not need to involve a dizzying amount of rework. The key tenet is centralizing and automating. Equipping infrastructure engineers, compliance experts, and auditors with a singular source of truth that makes intelligent, dynamic infrastructure decisions automatically based on compliance needs is the key to meeting, scaling, and maturing compliant infrastructure.

Five steps to mature and scale compliant infrastructure:

1. Migrate to a centralized module repository with a “write once, use many” mentality for cloud building blocks.

The repository should contain directories for specific resources or blocks of resources rather than application-specific services. For example, create a general AWS Relational Database Service (RDS) repository module instead of an application-specific RDS service module. Engineers should have the flexibility to pass any input they need to effectively use these modules in their applications.

2. Invest time into working with compliance teams to gather requirements for the target framework(s).

This step is a meeting of the minds to establish engineering requirements to meet compliance controls. Ask questions such as:

• What geolocation/data center requirements do you have?
• What are the encryption requirements for different types of persistent storage or communications?
• What data storage/lifecycle requirements will I have?

The result of this step is a set of stories or backlog tasks for the engineering teams to implement. Spend the time to map requirements into existing or new modules. For example, think about if a requirement applies to all data storage or just databases, effectively scoping the change.

3. Create “smart” modules.

What do I mean by smart modules? When refactoring modules based on requirements and tickets, think about the changes in terms of contextualized automation. The new modules should be designed and implemented to make compliance decisions automatically. A developer interfacing with the end module should only need to make infrastructure decisions pertinent to their application use case. This relieves the burden from sprawled teams’ developers to maintain compliant code. There are a few techniques that I recommend in making Terraform contextual decisions automatic:

• Create a new compliance variable called var.compliance_framework. Implement ternary operator logic in a local variables block to automatically set module inputs based on the compliance framework passed in.
• Make use of dynamic statements, switches, and data sources wherever possible. The broader coverage that the module can handle, the more equipped it will be to scale elegantly with compliant and non-compliant applications.
• Leverage validation conditions and Terraform 1.3’s new optional fields in variable definitions. It’s important to investigate resource defaults as part of this process to ensure that nulls or empty strings are passed properly as part of the default behavior and avoid creating a redeployment of an existing resource due to a change.

4. Move to a release-based software development lifecycle (SDLC) for Terraform modules to avoid cross-cutting dependencies and propagating errors.

Availability and up-time of production environments are core missions for infrastructure teams. Making significant redesigns of modules creates the potential for breaking changes introduced to a production environment. To mitigate this possibility, move to a release-based SDLC for centralized compliant modules. This will help infrastructure teams avoid propagating issues across multiple environments or regions and allow for testing and pinning to specific or older versions of modules until identified errors can be fixed.

5. Routinely re-audit usage of modules.

With a single source of truth for compliant code that automatically hardens based on the compliance framework, compliance experts can perform their reviews in one place instead of across sprawled repositories. As frameworks change, these changes can be made and audited from within this single repository.

The benefits of automating and centralizing compliance

This approach shifts the burden of compliance away from individual teams and into a centralized, automated, and scalable system. From a business and compliance side, compliance experts will be responsible for a single source of truth instead of diverging forks, branches, and repos, improving their review and audit efficiency and reducing the potential for human error. The “write once, use many times” design pattern coupled with automated compliance hardening opens compliance program scalability across the enterprise. New business units and compliance frameworks can be added organically with a faster time to value / MVP with reduced planning and compliance gap remediation required. A consolidated source for audits also simplifies the preparation and evidence-gathering process.

From a code and infrastructure side, careful planning of automated compliance hardening with ternary operator logic minimizes downtime and impact on existing production infrastructure. The ability to simply flip switches to turn compliance hardening on and off in new environments or regions alleviates the typical burden of planning for compliant infrastructure and enables limitless scaling of services using a common set of modules.

If you follow these steps, reducing the sprawl of compliance codified with Terraform will help mitigate the infrastructure side of this equation, increasing confidence in an enterprise’s audit readiness and ability to scale compliant infrastructure platforms to multiple business units.