Vulnerability Prioritization – Combating Developer Fatigue

Originally published by Sysdig on February 14, 2023.

Written by Miguel Hernández.

We are in early 2023, and we have over 2700 new vulnerabilities registered in CVE. It is still a challenge for developers to endure the fatigue of continual vulnerability prioritization and mitigation of new threats.

Our findings in the Sysdig 2023 Cloud-Native Security and Container Usage Report provide signs of hope for overburdened developers, as the data showed opportunities to focus remediation efforts on vulnerable packages loaded at runtime. Only 15% of high or critical severity vulnerabilities with an available fix are actually in-use at runtime.

Vulnerability management prioritization based on filtering by in-use packages enables teams to significantly reduce cycles spent chasing an endless pile of vulnerabilities. Despite increased adoption of shift-left security strategies to assess code early and often, organizations need runtime security, as part of a Cloud Native Application Protection Platform (CNAPP).

What is developer fatigue?

High-profile vulnerabilities and exploits, such as Log4Shell and Text4Shell, along with increased guidance from government organizations regarding cybersecurity, have caused many teams to heighten their focus on application security testing. Even with these high-profile vulnerabilities, there is little evidence of real progress in addressing this risk. In addition, development teams increasingly rely on open source software and third-party code, and with that comes the risk of exposure to both known and unknown security vulnerabilities.

A shocking 87% of images include a high or critical vulnerability, up from the 75% we reported last year.

When you view the data by number of vulnerabilities in images as opposed to number of vulnerable images, 71% of vulnerabilities have a fix available that has not been applied. Keep in mind, some images have more than one vulnerability. Organizations are aware of the danger, but struggle with the tension of addressing vulnerabilities while maintaining the fast pace of software releases.

Although the list of software vulnerabilities to fix seems endless, there is an opportunity to reduce wasted time and improve the efficacy of cybersecurity programs. Our research found that only 15% of critical and high vulnerabilities with an available fix are in packages loaded at runtime. By filtering out those vulnerable packages that are actually in-use, organizational teams can focus their efforts on a smaller fraction of the fixable vulnerabilities that represent true risk.

This is a more actionable number that can allay some fears around release decisions and focus remediation efforts, provided organizations use the relevant security capabilities.

Vulnerability prioritization as a mitigation method

Vulnerabilities are discovered in images each day. However, it’s not practical to fix every single one when you’re maintaining multiple workloads at scale. Successful, modern vulnerability management requires security teams to prioritize vulnerabilities based on the actual or real risk to their organization.

There are a number of inputs commonly used to prioritize vulnerability remediation work, which include:

• Common Vulnerability Scoring System (CVSS): specifies the severity of a known issue
• Exploitability: indicates if there is a known path for exploiting the vulnerability
• Fixable: identifies if there is a fix available to address the vulnerability

Addressing running, vulnerable packages with a known exploit should be the top priority. We found that our customers are proactive in fixing vulnerabilities that are exploitable and in packages loaded at runtime. When we combine multiple criteria of a vulnerability (fix availability, exploitability, and presence in a package loaded at runtime), what remains is 2% of the vulnerabilities found in the 25,000 images we analyzed.

Some vulnerabilities have an exploit available for attackers to use, but they do not have a readily available fix to mitigate potential threats. This small, but significant category affects security teams since they must still assess the risk of exploitable vulnerabilities and determine alternative mitigation strategies without Common Vulnerabilities and Exposures (CVE) patches or fixes.

When exploitable vulnerabilities must remain in your environment, one way security teams can ease the pain and reduce the risk of compromise is by implementing runtime security detections. Runtime protection is often powered by rules, but it should also employ a multi-layered approach that incorporates behavior anomaly detection and AI or ML-based detection. This approach improves detection and mitigation of zero-day exploits and yet-unknown threats.

Runtime protection mechanisms can also be tuned to detect novel threats that target vulnerable workloads in the unique environments of organizations. Detections can also be augmented with threat intelligence from threat research teams and regularly updated as new information or findings about behaviors become available.

We looked at the package types of more than 6.3 million running images to determine the four most commonly used package types.

• Java packages are the riskiest, representing over 60% of vulnerabilities exposed at runtime.
• Fewer than 1% of JavaScript packages are in-use at runtime.

Operating System (OS) packages were also risky, Most people use a base image because it’s easier than creating your own. Taking a look at our customer usage, we see that Red Hat Enterprise Linux (RHEL), which includes the Red Hat UBI (Universal Base Image), is by far the most popular at 46% of base images. This is up 10% year-over-year. This may be because RHEL has a long history of usage in the enterprise, and would be an easy choice as organizations move to cloud‑native workloads. Interestingly, only 16% use Alpine, a lightweight Linux distribution.

By using slimmed-down base images like Alpine, organizations can debloat their container environment by 97.5% and thereby reduce their attack surface. This will also reduce the number of OS vulnerabilities to fix, as only 8% of vulnerabilities are in OS packages loaded at runtime.

Conclusion

Companies are rapidly adopting containerized microservices, CI/CD, and on-demand cloud services to speed up innovation. However, the pace of change opens the door to risk as cloud sprawl and the complexity of cloud-native applications expose a lack of maturity in DevSecOps processes.

Vulnerability prioritization becomes an essential way to reduce or mitigate the stress and fatigue of our developer teams in the face of continuous vulnerability disclosure.

Finally, supply chain risk from misconfigurations and vulnerabilities has emerged as a major area of concern. Our research demonstrates that although there is awareness of required tools and the benefits of zero trust approaches, cloud security processes still lag behind the fast pace of cloud adoption.