Comments on the Extensible Visibility Reference Framework (eVRF) Program Guidebook

Originally published by Gigamon here.

Written by Orlie Yaniv and Ian Farquhar, Gigamon.

Editor’s note: Gigamon is very happy to see the CISA’s recent work on formalizing and structuring what visibility means and assessing its efficacy. As Zero Trust accelerates, visibility becomes a key focus in the process of assessing trustability and minimizing risk to enterprises and agencies that deploy it. We publish our feedback here in the spirit of openness and with the aim of encouraging comments and feedback that will benefit industry and government equally.

Gigamon appreciates the opportunity to comment on CISA’s “Extensible Visibility Reference Framework (eVRF) Program Guidebook.” As explained in detail below, we propose the following improvements:

1. Addition of a “telemetry assurance evaluation” as a fundamental component of the framework;
2. Explicitly noting the “defense-in-depth” approach that arises from multiple, overlaid forms of telemetry that are combined and analyzed to enable a better understanding of risk;
3. Allowing the use of coverage maps that extend beyond the MITRE ATT&CK framework; and
4. Explicitly stating that eVRF may be leveraged for better infrastructure design.

Gigamon delivers network visibility and analytics for digital applications and services across physical, virtual, and cloud infrastructure. Since 2004, Gigamon has been awarded over 75 technology patents and provides products and services to 80 percent of the Fortune 100 and most of the world’s top 10 banks, healthcare providers, technology companies, mobile operators, and government agencies. Our comments are based on our robust experience providing network traffic visibility to our customers.

Gigamon has used the term “visibility” since 2012. Our definition focuses on “seeing” all packets traversing physical, private cloud, public cloud, and hybrid environments and presenting packets or metadata derived from the packets to network-traffic consuming tools. Others, however, define visibility differently, which leads to a lot of confusion about the term. As such, Gigamon supports the definition on page 4 at lines 97–99 as a way of formalizing the concept and creating a common lexicon.

In Gigamon’s experience, many agencies have visibility gaps that expose them to risk (which is highlighted on page 1 at lines 30–31 and on page 4 at lines 123–125), and these gaps are far more extensive than realized. As such, the eVRF is essential for identifying and mitigating these gaps.

Assurance Evaluation for eVRF Infrastructure

In our opinion, the fundamental design concept is sound but needs to incorporate mechanisms for validating the integrity of logs, also referred to as “telemetry assurance.” This capability is necessary because:

1. All network components, including endpoints and endpoint detection and response tools, can be compromised and, as a result, generate inaccurate logs;
2. Sophisticated adversaries have demonstrated the ability to evade detection by modifying and spoofing logs (as opposed to stopping logs or changing the logging volume, which are louder actions that may be detected by agencies); and
3. The M-21-31 log integrity controls primarily protect the logging platform, not the components generating the logs.

To analyze telemetry assurance, Gigamon recommends four axes of evaluation as follows:

• Reliability: How comprehensive/accurate is the telemetry?[1]
• Evadability Resistance: How readily can an attacker evade detection in the telemetry produced?
• Resilience: Is it possible to detect the evasion?
• Stealthiness: Are the controls covert or detectable by a threat actor?

By way of example, consider host-based logging to a centralized SIEM. The host-logging capability maps to a sensor in eVRF terminology inside an observation point (the host) sending to a visibility surface (a SIEM) inside an agency’s on-prem network (the domain). Using the proposed criteria above, we evaluate assurance on logging as follows:

Criteria	Evaluation	Commentary
Reliability	Low/Medium	See footnote on “log attack chain.”
Evadability Resistance	Low	Disabling/degrading/spoofing logging is a standard attacker TTP and can be done from inside the workload.
Resilience	Low/Medium	While techniques exist to detect attacks on logging, their efficacy is low, producing both false positives and negatives. [2]
Stealthiness	Low	Logging is configured inside the workload. An attacker who has compromised the workload will be aware that it is logging.

Using the above criteria, host-based logging is not a high-assurance mechanism. Logging nonetheless remains critical and necessary. However, in using logs as a visibility mechanism to detect adversaries, log assurance needs to be evaluated and enhanced as appropriate. Unless the integrity of logs is validated, they cannot be trusted and relied upon for threat detection and incident response.

Another example from Gigamon’s area of expertise is both packet-level network traffic and network traffic metadata. Consider an IaaS cloud environment as the domain and the visibility surface as intra-workload network traffic visibility.

Let’s consider two alternative techniques (sensor types) for gaining that packet-level visibility: using a service on the cloud provider itself (e.g., Amazon’s VPC Traffic Mirroring) or a workload-based agent that replicates all traffic to the monitoring platform.

SaaS Network Visibility delivered via workload agent:

Criteria	Evaluation	Commentary
Reliability	High	Delivers all traffic to and from this workload
Evadability Resistance	Low	Agent can be disabled by an attacker who has compromised the workload
Resilience	Low/Medium	While cessation of traffic streams can be detected, a very sophisticated attacker could theoretically mask or filter out their traffic
Stealthiness	Low	The running agent is visible to an attacker who has compromised the agent

SaaS Network Visibility delivered via cloud platform traffic mirroring capabilities:

Criteria	Evaluation	Commentary
Reliability	High	Delivers all traffic to and from this workload.
Evadability Resistance	High	Configured in the cloud management system, not disruptable by an attacker on a monitored workload.
Resilience	High	Attacker would have to compromise the cloud management system. The compromise of the workload alone would not allow this to be disabled.
Stealthiness	High	Invisible from the workload.

As highlighted, the same type of visibility on the same domain delivered by different varieties of sensor technologies can have very different levels of assurance.

A key benefit of eVRF is not only the analysis of visibility coverage but also to inform procurement decisions. Certain options may be more expensive but may provide visibility at a higher level of assurance and trust, which are critical for improving cybersecurity outcomes.

Recommendation: Consideration of the assurance level provided by a particular type of visibility should be determined. Considerations should include (1) reliability, (2) evadability resistance, (3) resilience, and (4) stealthiness. Formalize this process as a part of the structure, potentially adding the step at the coverage comparison layer to identify low-assurance visibility, and support it with layering of visibility to ensure adequate coverage.

As a further observation, when evaluating visibility coverage and using CISA’s analogy, the structure needs to know where the awnings and shadows are that allow an attacker to hide. Assurance evaluation will assist, and low assurance mechanisms can be improved, by combining multiple observability surfaces. While this concept is implicit in the overlaying visibility coverage, we recommend explicitly stating it to reinforce the value of the concept.

Recommendation: Explicitly note that different types of visibility can support a cross-checking of assurance and trustability as a Zero Trust/defense-in-depth philosophy.

Section 1.2, “Benefits of eVRF” (lines 22–35): The deployment of this methodology will drive better architectural decisions at the design phase. Traditionally, visibility has not been a design requirement of many IT architectures, and driving a change where it becomes a requirement will significantly improve cybersecurity outcomes.

Recommendation: Add the benefit of driving architectural designs to support visibility as a clear benefit of the eVRF.

Possible Other Uses

Gigamon fully appreciates that the scope of this document is improvement in cybersecurity (EO 14028 being explicitly mentioned in line 18), but we observe that the structure is equally applicable to many other use cases. One closely adjacent use is compliance measurement. Gigamon encourages the expansion of the structure to acknowledge other possible visibility coverage maps beyond the MITRE ATT&CK ones.

Specific agencies may also have atypical requirements that go beyond the controls in the MITRE ATT&CK matrix, such as mission-specific requirements. This is another reason we believe that opening the eVRF framework up to other visibility coverage maps produced by CISA, other agencies, or third parties improves the framework.

Recommendation: Explicitly allow the use of other coverage maps within the structure, targeting both other use cases and mission-specific requirements.

Conclusion

Gigamon is supportive of this initiative and looks forward to working with CISA and FCEB agencies to move this forward.

---

Notes

[1] While logging is an essential component of any security strategy, logs are inherently vulnerable to compromise, and sophisticated threat actors often disable or slow logging capabilities and/or modify logs. The veracity of logging for security outcomes depends on a chain of assumptions that mature adversaries will exploit and break. Specifically, these assumptions include the following:

1. The digital event must be perceptible in the environment running the software
2. The digital event must have been anticipated in the development of the software, and code to generate a log event included in the software
3. That logging event must be enabled and logging properly configured
4. The log event must be transmitted securely to the SIEM or security analytic capability in a way that it is not lost, disrupted, misrouted, or spoofed
5. The log event must be ingested into the SIEM and processed into an informational taxonomy that supports understanding the mapping of the log event to the digital event
6. The significance of that digital event must be understood
7. That understanding must be related to an operational capability that can triage, assess, and remediate the risk

If an adversary designs an attack that sidesteps any assumption, the event will not be logged or detected without an alternative means of detection.

[2] Gigamon acknowledges the log evasion controls in M-21-31 but notes that only log cessation (page 7) would be detected, and the integrity checking mechanisms that would detect changes