Situational Awareness for Detection and Analysis: Go with the Flow

Originally published by Netography.

Written by Martin Roesch, CEO, Netography.

When we look at the threat continuum, the preparation of the assets and infrastructure in a modern network to resist an attack, including discovering, configuring, and hardening, requires major investment in tools and technology to establish a defense. Despite those efforts, operators will inevitably find themselves required to go further down the attack chain to deal with compromise and exploitation. To do this effectively in an Atomized Network, security, and network operators need more specificity and situational awareness to do their job.

Situational awareness is difficult to achieve without broad coverage of a networked environment and real-time, high-quality insights accessible to users. Realizing this capability means the delivery system must be architected to provide visibility across an entire enterprise network, deliver it in real-time and at scale, and provide analytics with near real-time access to investigations. While these insights into activities and security-relevant events are available from deep packet inspection (DPI) technologies and analysis of cloud logs, a less limited view delivered by a broader approach is needed in the modern enterprise.

Architectural mismatch

Atomized Networks are especially problematic for traditional DPI-based methods to provide visibility and awareness. This is especially true for the dispersed and ephemeral nature of multi-cloud environments. DPI technologies can only see the subset of traffic that is directed past their sensing interfaces and are usually delivered in hardware-based or virtual appliances or possibly even sidecar loadouts in the cloud. These delivery methods are typically also running a set of detection logic of either deterministic rules or machine learning models to search for signs of active threats. If we are using DPI-style methods that look at the traffic passing very specific points on the network and have multi-thousands of rules to detect very specific attacks, then there’s a lot we miss, including east-west traffic, misconfigurations that may open the network to compromise, post-exploitation persistence activities, lateral motion, and literally any area of the network where budget or access prevent deployment of capability. Getting inspection technology where we need it and when we need it is very difficult in dispersed environments where data, applications, devices, and users are everywhere.

The dispersed and ephemeral nature of the cloud presents additional challenges because workloads can spin up in a matter of seconds. Deploying DPI sensors that are sidecar loaded or virtual appliances require managing license keys, curating policy, and integrating management infrastructure to operate. Sensors may have clear text access to the encrypted traffic but if not, then a separate piece of hardware with its added cost and complexity is required to do decryption. And still, this method only sees what is right in front of it instead of the bigger picture, and it may not have context about the environment apart from the application that it is watching.

Getting to situational awareness

Situational awareness versus pure inspection in the service of threat and exploit detection is fundamentally different and, we believe, a better approach for the atomized world.

Situational awareness requires two capabilities: one, leverage continuous observation of the entire environment, as opposed to choke point monitoring and detection, and two, ask any question at any time about activities observed in the environment. In a detection-centric model, we have one chance to detect an event and if we don’t detect it in the few milliseconds it occurs, then it is gone. Additionally, because it’s not feasible in most networks to store all of the traffic for later retrospection, the moment where the event occurs is the one and only moment that detection can happen. If it’s not detected then all history of its occurrence is lost and there is no way to look back later.

Generating appropriate context about the operational environment ahead of an event is important because networks can change in the normal course of business, especially in ephemeral environments, or even be changed by an attacker. Security operators generally don’t respond in seconds but in hours, so the data that reflects the composition of the network needs to be collected either before or at the time that the traffic or events are generated – not afterward.

Situational awareness should deliver detection and analysis across a spectrum ranging from discovery and mapping to operational governance – informing of noncompliance and configuration drift – to compromise detection and threat hunting. If an event is missed or new information about attacks or compromise is brought to light, look at what’s been observed in the past and then get smarter about what has happened.

It's important to map the entire environment so that as flow data arrives, you can enrich it with operational context to provide a contextualized picture of what is happening. It’s a radically different approach from anything that has come before and it’s what’s needed right now.