Will the Cloud Kill Security Agents?

This blog was originally published by Sysdig here.

Written by Anna Belak, Sysdig.

The “agents or no agents” debate is ancient and eternal. Every decade or so, we go through another round of “agents are terrible, let’s end them” and “we need more visibility and control to secure the system, maybe we’ll call it a ‘sensor’ this time.” We ultimately always land on the same conclusion. There are no silver bullets. Today, the debate is alive and well because cloud is the new frontier, so surely agents are dead this time? We don’t think so.

Agentless is great to get you started in cloud security. If you’re not willing to accept blind spots, you’ll need agents and other approaches too.

Cloud environments are nice because they offer a uniform, API-based control plane. This is easily accessible to agentless security methods for identifying many types of problems. Plus you get a quick onboarding experience, so you can feel like you’re winning right away. But in security, the whole story is never very simple. Your ability to sleep at night depends much more on a truly layered defense than a quick win on day zero.

When evaluating security tools designed for the cloud, you will find agentless, agent-based, and those that use combined approaches. Even if your team is just getting started with the cloud, you should consider how the tool will accommodate you as your environments expand and your processes mature.

The table below shows some examples of cloud security use cases and whether an agentless or agent-based approach is more suitable for them.

Agentless Approaches to Cloud Security

Agentless cloud security solutions aim to be unobtrusive and easy to manage while providing visibility into the security posture of cloud environments.

They work by inspecting metadata, leveraging APIs, and using other indirect methods to scan your environment for vulnerabilities and misconfigurations. Agentless methods can provide some visibility into systems where agent instrumentation is impossible or doesn’t make sense, such as serverless compute instances or other cloud-native services.

An agentless security tool isn’t magic. It needs visibility into your cloud accounts. This is typically achieved by defining a special IAM role for the third-party vendor that grants, usually read-only, access to the assets you want scanned.

Many agentless tools market themselves as “continuous monitoring.” In this context, “continuous” usually means “periodic,” as in every day or every hour. “Monitoring” usually means “scanning.” The tools will compare current state with the last known state and report on any changes between the two. However, they do not have real-time visibility, so they may miss information about intermediate states of the system between scans. This aspect can create challenges for dynamic and short-lived workloads, which may spin up, perform their task, and disappear having never been detected by the tool.

Agentless tools are more powerful in the cloud than they were on-premises due to the programmatic infrastructure that the cloud provides. These tools can address multiple use cases without imposing any performance overhead on your workloads, and demanding very little management overhead from your teams.

Agentless Pros:

• Easy to deploy
• Minimal management overhead
• Non-intrusive to the workload
• Can be deployed in systems that cannot accommodate agents
• No or very low workload performance impact

Agentless Cons:

• No real-time workload monitoring capability
• Limited visibility into the system
• Relies on cloud provider capabilities, so doesn't work on-premises
• Cannot execute active response actions

Agent-Based Approaches to Cloud Security

One indisputable advantage of agent-based security approaches is that because they sit on the system, they can see what is running on the instrumented machine in real-time, all of the time. When that machine is in the cloud, it actually becomes easier to fetch relevant context and enrich the generated alerts very quickly.

Certain types of telemetry, like system calls, can only be collected locally. Thus, you must deploy an agent if you want access to this data. Furthermore, agents can facilitate immediate response at the workload, such as blocking the execution of malicious processes or capturing telemetry and metadata about potentially nefarious activity. Modern agents are designed to be light-weight and leverage technologies like eBPF to reduce their impact on the host system. This is why all major endpoint protection systems are agent-based.

You do have to deploy the agents to the target machines and occasionally manage them. Today this work is simplified and accelerated by infrastructure-as-code and orchestration systems like Kubernetes, so you may only be a YAML file or two away from that quick win.

Smarter agents can perform analytics at the workload. This makes it possible to quickly enrich data, decide which data to forward on to central systems like SIEM, and take immediate enforcement actions when known-bad signals are detected.

Agent-Based Pros:

• Deep host-based visibility
• Can run anywhere if OS is supported
• Real-time detection for faster time to discovery
• Rapid incident response and containment

Agent-Based Cons:

• Management overhead
• Performance impact
• Does not work for systems that cannot accommodate the agent install

To Agent or Not To Agent

Let’s explore whether agents are necessary or optional in a handful of cloud security use cases. We will consider four distinct categories across cloud and containers: vulnerabilities, configurations, permissions, and runtime security.

Vulnerabilities

The cloud vulnerability management story benefits hugely from agentless approaches, alleviating many on-premises pain points at once. However, where agents can complement and add value to agentless vulnerability management is by providing context for more effective risk-based prioritization and/or mitigating flaws that cannot be remediated right away.

• Asset discovery and inventory management: In cloud environments, it’s easier to track assets due to the API-driven nature of the whole thing. In other words, the cloud knows what you put in there, and it will happily tell you this, without any need for an agent. However, certain types of assets, like containers, remain elusive even in the cloud, and an agent is better suited to keep track of them.

• Vulnerability assessment is the process of enumerating the known vulnerable components in your environment. Modern cloud security tools can achieve this by pulling the relevant data from cloud-native sources like virtual machine images or snapshots, removing the need for network scans or agents. Born-in-the-cloud vulnerability assessment approaches use cloud APIs to inspect virtual machine images, container images, storage snapshots, and other data sources.

There are two areas where agent-based approaches can add significant value over agentless approaches to vulnerability management:

• Risk-based vulnerability prioritization is the method of determining which problems to fix first by considering how much actual risk they pose to your organization. Agentless approaches can consider facets such as CVSS ratings, and even attributes such as the asset’s network exposure, its inherent business value, and the threat intelligence surrounding the vulnerability in question. However, agent-based approaches have a significant advantage in one area that can have a massive impact on prioritizing software vulnerabilities. An agent can enumerate running processes within each machine, and prioritizing vulnerabilities based on what packages are actually being used versus those that are included can make remediation prioritization radically better.

• Vulnerability mitigation is any action you take to reduce the potential impact of an exploit while the vulnerability remains in your environment. Often, this is a temporary measure because a fix is not available or cannot be immediately deployed. Some mitigations, like virtual patching and configuration changes, don’t require agents. However, agent-based mitigations like application control or detection and response tools can also be very effective, especially when a preventative compensating control may be too disruptive.

Configurations

Configuration management is even more critically important in the cloud than it was in the data center. In the data center world, many layers of security controls stood between the scary outside world and some misconfigured server deep inside your production environment. In the cloud, a large proportion of breaches occur due to fairly simple configuration errors.

• Cloud management and deployment configuration refers to how your cloud services are configured. This is the realm of exposed storage, excessive permissions, disabled logging and monitoring, and so on. Agentless tools help here by checking your configurations against best practices and compliance requirements, such as the CIS AWS Foundations Benchmark or PCI DSS, and there is no need for an agent in every workload.

• Cloud workload configuration includes host hardening, Kubernetes and container security, and other workload-specific best practices, such as CIS benchmark compliance for those particular systems. Static assessment of workload configuration does not require an agent.

While agentless approaches can effectively validate cloud configurations statically against best practices, agent-based approaches are much better suited for detecting real-time drift and minimizing time-to-detection.

• Configuration drift detection lets you know when something has changed in runtime relative to how it was initially defined. Here, an agent may be necessary if time is of the essence. Agentless tools can report on drift as frequently as they are able to check in on your environment, but agents have real-time visibility and can immediately alert you to anomalous activity.

Runtime security

Runtime workload protection is necessary because prevention fails. Even if we built the most robust preventative controls, they would not be perfect. Sometimes, the bad guys get in or our teams simply make mistakes, and we should have detective controls to catch the problem quickly and minimize the impact. Not all detective controls work in real-time, but the ones that do require an agent.

• Malware prevention, especially advanced techniques like fileless or mutating malware, are most effectively addressed via agent-based methods. Agentless approaches can identify and remove bad files but will miss real-time malicious activity that occurs between scans.

• Real-time threat detection and response is the ability to detect an active attack in real-time and hopefully contain it. These detections require agents and can be based on behavior analytics, command sequences, certain TTPs, or other methods of proactive contextual alerting with the intent of interrupting the attack chain.

• Incident investigation can involve agentless sources such as logs, network forensics, file system analysis, and so on. However, if an agent was present on the compromised system and was able to capture real-time data about the attacker’s activity, it can provide a lot of additional clues about what exactly happened.

• Advanced persistent threats are highly skilled malicious actors, often associated with nation states, who are targeting organizations with very specific goals. These actors are very good at evading controls and covering their tracks. Agent-based monitoring gives organizations a chance to catch nefarious activity in real-time, before evidence of tampering is destroyed.

Conclusion

Agentless approaches are effective for compiling an inventory of cloud services your team is using, identifying known vulnerabilities in software, and detecting threats based on logs.

Real-time detection of runtime threats, malware, and advanced persistent threat detection requires an agent. Once you detect a threat, the detailed activity record and context an agent provides is critical for incident response, containment, and forensic investigation. You can check the box on many compliance requirements without an agent, but to effectively manage security risk you will find yourself using both approaches.

Both approaches are useful and offer different advantages. Agentless deployments are easier, require minimal management overhead, impose little to no performance overhead, and can accommodate systems that cannot incorporate agents. Agent-based approaches provide much deeper visibility that facilitates more comprehensive context, real-time detection, and are necessary to go beyond detection to enable faster incident response, containment, and investigation.

We have seen this movie before, and we know how it ends. The original vulnerability assessment network scans were “agentless.” Unsurprisingly, they weren’t able to extract enough data from the systems to give teams a full picture of what they needed to fix. Then, vulnerability management agents were invented to improve the situation, and every self-respecting scanner became “hybrid.”

The tide comes in, and the tide goes out.