You Can’t Detect What You Can’t See – In Real-Time. The Modern Cloud Security Edition

This blog was originally published by Blue Hexagon here.

Written by Arun Raman, Blue Hexagon.

There is an old maxim in cybersecurity: “You cannot detect what you cannot see.” While we hold the need for security visibility as a self-evident truth, there are implications for a cloud security architect who may be faced with what seem like unavoidable compromises while trying to reify the spirit of the maxim. CISOs and security practitioners need actionable visibility and insights into their public cloud environment spanning, e.g. AWS, GCP, and Azure — this can be made possible by a set of carefully implemented design choices that form the bedrock of a multi-vector, comprehensive, and consistent actionable visibility architecture for cloud security. In this blog, I will elaborate on these design considerations, which I hope will be of use to the cloud security practitioners.

Figure 1: Cloud security visibility must be multi-vector, comprehensive, and consistent in order for security teams to operate effectively at cloud scale and speed.

Multi-Vector Security Visibility

All computing, particularly cloud computing, can be described as a composition of four basic services – compute, storage, networking, and the control plane. Lack of visibility into any of these can result in gaps in your security posture that can and will be eventually exploited by adversaries. A security platform that provides cross-cutting visibility across the four essential services is said to be multi-vector (see Figure 2).

Figure 2: Multi-vector security platform provides cross-cutting visibility and threat defense across network, storage, workloads, and the control-configuration plane.

Comprehensive Security Visibility

Cloud service providers offer several compute fabrics with different levels of management and shared responsibility. Workloads typically run on some combination of the following:

• Infrastructure-as-a-Service (IaaS):

Compute: E.g. baremetal, AWS EC2, GCP Compute Engine, etc.

Storage: E.g. AWS EBS, GCP Compute Engine, Azure Disk Storage

• Platform-as-a-Service (PaaS):

Compute: E.g. AWS ECS, AWS EKS, GCP Kubernetes Engine, Azure Kubernetes Service, AWS Lambda, GCP App Engine, etc.

Storage: E.g. AWS S3, GCP Blob Storage, AWS EFS, GCP Filestore, etc.

The diversity of cloud resource fabrics immediately informs us that comprehensive security visibility must be:

• Cross-IaaS and PaaS: Works across both IaaS and PaaS compute and storage services listed above
• Cross-Platform: Works across managed compute and storage platforms listed above
• Cross-OS (Operating System): Works across Linux, Windows, OS X servers
• Cross-Application: Works across variety of application stack technologies
• Cross-Cloud: Works in a similar fashion across the major public cloud providers

Consistent Security Visibility

By consistent security visibility, we mean that the desired visibility properties apply not just to some workloads or some accounts, but rather apply uniformly to:

• Every Workload
• Every VPC
• Every Account
• Every Region
• Every Cloud

Given the diversity of resource types discussed in the previous section, achieving consistent visibility is often non-trivial and relies heavily on automation as I’ll explain further in the next section.

Consistent security visibility is not just a spatial property but is also a temporal property

Consistent security visibility is not just a spatial property that applies only to different types of resources in the cloud resource hierarchy but is also a temporal property that must be architected for the lifetime of the associated resource type. E.g. A pre-infected container image may beacon out to a remote C&C server when the image is launched. Detecting such behavior requires networking visibility to be active when the container launches. The same running container may download and run cryptomining malware during the course of its execution, detecting which requires multi-vector visibility to be active throughout the container’s lifetime.

Contextual Security Visibility

Security visibility devoid of timely context is of limited value, and nowhere is it more pronounced than in the cloud! Consider a network security finding, a C&C communication alert perhaps, associated with an internal IP address in your cloud environment. If this alert is analyzed even minutes after the fact, attempts to gather context may be futile since the ephemeral nature of cloud workloads means that the IP address could have been associated with a Kubernetes container pod at the time of the event and be associated with a VM at the time of subsequent detection. Cloud providers do not provide logs tracking all types of resource associations of interest, and attempts at ex post facto correlation can be error prone. There is truly no substitute for gathering relevant context in real-time, and doing this at scale in a timely manner requires both AI and automation.

There is truly no substitute for gathering accurate and relevant context in real-time

Equally important is contextualizing security visibility of the network, storage, and control plane with cloud-native abstractions of the impacted workloads and applications. In practice, this means that, as much as possible, every data point, observation, alert, or incident must have the associated application tags and metadata as well as the relative importance and impact on the broader security posture of the organization. Such contextualization effectively prioritizes the findings for the security teams to act upon.

A Formula for Multi-Vector, Comprehensive, Consistent Visibility

I propose the AAA formula for architecting true multi-vector, comprehensive, and consistent security visibility across your cloud environment: automation, AI-driven, agentless.

Figure 3: The AAA formula for architecting true multi-vector, comprehensive, and consistent security visibility across your cloud environment.

Automation

Cloud providers such as AWS, GCP, and Azure have well-defined event buses where resource lifecycle events are published. Triggered by these events, put in place automated guardrails that enforce desired security visibility for the resource.

Example 1

DevOps creates a VPC in a new region for testing multi-region support of their application. With legacy DevOps / SecOps interaction practices, this may result in friction and delays to get internal approvals and may require the security team’s manual intervention to configure security visibility into the new environment. Contrast the above with the proposed approach that automatically detects (via the event bus) that a new VPC is being created, triggers the set up of automated guardrails for network-based visibility through AWS VPC Traffic Mirroring (or GCP Packet Mirroring, or Azure VTAP), and ensures that SecOps has complete packet-level visibility into the newly created VPC without any human intervention.

Example 2

DevOps creates a new blob-storage bucket for storing and processing documents uploaded by customers. With legacy DevOps / SecOps interaction practices, this may result in the security team’s manual intervention to determine properties such as possible public access (both at the granularity of the bucket and individual objects), register the bucket for periodic antivirus scanning, and lock down access to the bucket to specific roles associated with the document processing microservices. Contrast the above with the proposed approach that automatically detects (via the event bus) that a new bucket has been created, installs guardrails disabling public access unless explicitly overridden, automatically triggers malware inspection of payloads on create/modify operations, and checks for malicious or suspicious access from IAM or non-IAM entities.

AI-Driven

Cloud environments at enterprises routinely generate over a billion security-relevant data points every single minute. AI helps transform this huge volume and dimensionality of data into concrete contextualized security findings that humans can reason about, and in that sense AI has a force multiplier effect on Security and SecOps teams – it helps them focus on higher-value work that they can do more of and do it faster. This frees up the security team to focus on higher value work and upskilling, and keep up with workloads operating at cloud scale and speed.

Figure 4: Security visibility driven by Deep Learning AI force-multiplies your security team, and detects unknown threats that pose the greatest risk to your cloud environment.

AI, more specifically Deep Learning AI, can find the unknown–unknown which poses the greatest risk to enterprises. Legacy signature-based and sandbox-based defenses have failed us. Today, even sophisticated enterprise security teams expend a lot of effort and energy hunting for threats and they have to make critical security decisions in real-time — Deep Learning AI can greatly improve the efficacy, speed, scale, and timeliness of such decision-making and in the right contexts automate it entirely with autonomous response capabilities.

Agentless

We have written previously about the numerous benefits of agentless security inspection here. In a nutshell, rather than attempt to deploy security visibility agents on each and every workload (cumbersome at best, infeasible at worst), you can use cloud-native functions and APIs to gather the most valuable data from a security standpoint out-of-band effectively creating an orthogonal security visibility plane. These cloud-native functions and APIs include event triggers (think EventBridge and CloudWatch), control plane APIs (to access cloud configuration information, contextual workload metadata, take snapshots, etc.), traffic mirroring (to access full L3-L7 network packet data in real-time), and the broader set of cloud provider service APIs. A key defining characteristic of such an approach to cloud security is that security teams now do not need to tightly couple or interface with the typically much larger Developer and DevOps teams to instantiate and maintain security visibility.

With the agentless security visibility plane, a security team can deploy security visibility with zero downtime or changes to existing or future workloads

---

Read other interesting blogs by Blue Hexagon at http://bluehexagon.ai/blog.