Egress URL Filtering: The Most Important Cloud Security Control You’re Probably Missing

Originally published by Valtix.

Written by Vijay Chander, Valtix.

As we work with enterprise cloud security architects daily, it’s abundantly clear that one of the top priorities in 2023 is how to standardize security policy enforcement through improved network architecture across project teams and accounts. We hear that secure cloud networking is not just a nice to have; it’s a must-have for these architects in 2023.

But what are the most important controls to enable the security of cloud networks?

Again and again, URL filtering of cloud egress traffic, along with much more well-known FQDN filtering, comes to the top of the list. This blog will explore why.

What’s cloud egress traffic URL filtering?

URL filtering works by analyzing web traffic and filtering it based on the URLs (e.g. https://domain.com/URL) that are accessed. This involves identifying the URLs accessed by your applications and then comparing those URLs to a list of known malicious or potentially harmful URLs or known good URLs. If the URL that is accessed matches a URL on the harmful list, the traffic is blocked, and the connection (possibly malicious) is prevented from connecting. If it matches an approved one, then the connection is allowed. More on why this is important later…

URL filtering is a well-known technique for the security of ‘user’ endpoints. Most enterprises have a mature URL filtering technology as part of a suite of end-user security functions. What’s new is applying this control to cloud deployments to protect workloads and data from command-and-control (C2) and exfiltration. And if you’re running a Virtual Desktop Infrastructure (VDI), the need should be even more apparent.

Domain or fully qualified domain name (FQDN) filtering works similarly but focuses only on the destination domain. Just because a solution delivers FQDN filtering doesn’t mean it can provide URL filtering (i.e. solutions like Squid, AWS Firewall, and Aviatrix don’t provide this critical capability).

URL filtering requires TLS decryption since the URL is part of the TLS-encrypted HTTP traffic. FQDN (domain) filtering doesn’t require decryption, although it does require category-based domain intelligence that most solutions don’t provide.

URL and domain filtering work on a very simple idea. Allow or block traffic to a destination by checking against…

• Customer-provided lists of known good sites (domains) and URLs
• Domain and URL categories that you approve

A great example of a domain where you also need to care about the URL is github.com. Whether the URL is github.com/MyOrganization vs. github.com/MaliciousSite is a pretty important detail when implementing a secure egress strategy. Clearly, FQDN filtering will not help in this case. You need to filter based on URL.

Domain and URL category classification is provided by a service built into your security platform that can in real-time classify any domain/site and URL. This means that you don’t have to explicitly list the specific ones you want in your policy out of the millions of domains and billions of URLs. This is especially useful for VDI use cases. So you can:

• Block all malicious categories like malware, phishing, botnets, and so on.
• Allow only approved pre-defined categories like Financial Services, AWS Cloud Services etc.
• Allow only approved customer-defined categories: My Payment Processors, My Third-Party Vendors

Why does it matter?

Without domain and URL filtering, the only way to stop an attacker inside your network is to add each known good IP to an access control list (ACL). But this is not practical as the IP addresses of commonly used services and sites are substantial and dynamic. Caching at the edge makes this even more challenging, as that can result in an explosion of IPs.

Both these controls are also required by most security compliance standards and guidelines, such as PCI DSS and SOC2 Type2. For example, PCI Requirement 1.2.1 states, “Restrict inbound and outbound traffic to that which is necessary for the cardholder data environment, and specifically deny all other traffic.”

This really stops an attacker or malicious insider from…

• Connecting to their command and control (C2) for persisting an attack inside your network
• Exfiltrating data to a site that they control

What’s special about URL filtering

While domain-based filtering will ensure that known malicious ones are blocked, it’s often a challenge to use common public services that can potentially host malicious actors. For example, you may be using the following services, but they could also be used by the attackers. So domain-based filtering provides a good baseline, URL filtering is a necessity to prevent malicious actions.

The key point from this… Imagine an attacker able to set up a legitimate-looking GitHub repository to exfiltrate data. Given the likelihood that you already have many legitimate GitHub connections, this activity would go completely undetected.

Why is URL Filtering so important? It’s the last line of defense in the case of a reasonably sophisticated attacker who manages to bypass other defenses. How do you implement URL Filtering? That’s where security inside your cloud networks comes in. More on that in the next part of this series.

What are the essential requirements for your URL and domain filtering controls?

• Support for custom lists and categories of domains and URLs
• Built-in categories for real-time lookup of 100s of millions of domains and 10s of billions of URLs, both good and bad ones
• Support for TLS decryption at high performance and low latency to perform URL filtering without hindering your applications
• Ability to apply domain and URL filtering based on the workload’s identity, i.e. using its cloud-assigned tags: web, backend, database, pci, prod, test, staging, business-team, app-type
• Adaptability to specific network, data, and/or app requirements.
• And finally, the ability to deploy the infrastructure to do all of this in minutes, not hours, and route all of the network traffic to the appropriate point of inspection.