Your Guide to FedRAMP Pen Test Guidance 3.0

Originally published by Schellman here.

Written by Josh Tomkiel, Schellman.

For the first time since 2017, the FedRAMP Project Management Office (PMO) has updated the Penetration Testing Guidance document.

For Cloud Service Providers (CSPs) seeking FedRAMP Authority to Operate (ATO), that’s important news. What will be more important is ensuring you understand the changes. Through direct conversations and various surveys, Schellman’s Pen Test Team provided feedback to the PMO—feedback this update took into consideration. During that time, we gleaned considerable insight that we’d now like to pass on to help you.

In this article, we’ll do two things: we’ll provide a rudimentary summary of what’s new in the guidance and we’ll also dive more technically into the six attack vectors that must be performed. Whether you’re looking for a brief overview of what’s new or a detailed deconstruction, you’ll have a better idea of what is required for a FedRAMP pen test.

What’s New in the FedRAMP Pen Test Guidance 3.0

Here’s a high-level overview of the update:

• Renamed attack vectors
• The Internal network attack vector is now merged with the External network attack vector, with an updated focus on the FedRAMP authentication boundary and the FedRAMP boundary (Internet-facing and internal network) itself
• Clarification on testing client-side applications
• All testing must now be performed in the production environment (no staging/QA)
• More details on what is expected of CSPs regarding Attack Vector 1: External to Corporate (Phishing):
  • You must allow phishing attacks past all technical controls in place to see how users would respond.
  • Any data submitted within the campaign, real or not, is to be considered a failure of the test.
  • You must provide security awareness training and password rotation for any users that fall victim to the phish at the end of the campaign.

Taking all that into account, we’ll delve a little deeper into specific attacks and how they’ll work.

As a reminder, six different attack vectors are in scope of a FedRAMP penetration test, each designed to simulate a different, realistic threat. To dive deeper into each attack vector, keep scrolling or choose from the list below:

1. External to Corporate (Phishing)
2. External to CSP Target System (External and Internal FedRAMP Boundary)
3. Tenant to CSP Management System (Web Application)
4. Tenant-to-Tenant (Web Application)
5. Mobile Application to Target System (iOS and Android)
6. Client-Side Application and/or Agents to Target System

Attack Vector 1: External to Corporate (Phishing)

Attack Type: Execution of a social engineering (phishing) attack targeting your system administrators and managing personnel who may influence system administrators.

• It’s possible to expand beyond credential harvesting e-mail-based attacks—discuss different possible scenarios involving script or file execution during the planning stage.

How Does It Work: Your employees will be tested with a “worst case scenario”—a phishing attack has made it to their inbox, what will they do next? Will they fall victim to a sophisticated social engineering attack?

• You will provide a list of employee names and e-mail addresses to make up the target list.
• The phishing campaign must be allowed through any technical preventive controls such as e-mail filters or web content filtering proxies.
• Any data submitted within the campaign, real or not, will be considered a failure of the test.

Follow-Up Action: Upon completion of the campaign, You must provide security user awareness training to those employees who fell victim to the phish and provide evidence that their credentials have been rotated.

Attack Vector 2: External to Target System (External and Internal FedRAMP Boundary)

Attack Type: This attack vector includes two different types of network assessments:

• External threats: An Internet-based attack as an uncredentialed attacker attempting to gain unauthorized access into the FedRAMP boundary.
• Internal threats: Attempted exploitation of weak permissions/access controls and poor customer separation measures—e.g., improper network segmentation and poor implementation of security controls—as well as abuse of system services.

External Threats

How Does It Work: As an unauthenticated attacker on the Internet, the Pen Test Team will perform active reconnaissance, vulnerability scanning, and manual testing to identify and exploit any vulnerabilities on Internet-facing hosts within the production FedRAMP boundary.

During testing, all external endpoints are understood and all passive or active blocking security devices—such as web application firewalls and or software-based security controls—are bypassed.

Internal Threats

How Does It Work: The Pen Test Team will attempt to gain access to the FedRAMP authorization boundary as an unauthenticated user before connecting to the FedRAMP boundary to look for misconfigurations or vulnerabilities within the boundary itself.

Achieving these objectives will likely mean unique approaches that depend on the CSP. That being said, there’s a two-part process required for simulating an internal threat attack against the FedRAMP boundary:

• Part One: The Pen Test Team will attempt to breach the access controls used by your authorized employees to access the FedRAMP boundary, and there are some variances depending on how the boundary is currently accessed.
  • In the case of network-level access controls such as a VPN or reverse-proxy, the Pen Test Team will evaluate the control’s security posture and look for misconfigurations that would allow for unauthorized access.
  • Single Sign-On (SSO) providers and other identity management solutions will be assessed for weak permissions, privilege escalation, impersonation of authorized CSP employees, and whether Multi-Factor Authentication (MFA) can be bypassed.
• Part Two: You will provide credentials to the Pen Test Team so they can access the production FedRAMP boundary as an authorized CSP employee and assess inside the internal boundary for intentional, unintentional, and any other applicable threats. More specifically, they will:
  • Confirm access to the internal network and begin testing from the production subnet.
  • Attempt to identify and exploit vulnerabilities or misconfigurations within the FedRAMP authorization boundary.
  • Assess the controls implemented to prevent an attacker from pivoting from one internal network segment to another.
  • Work with you to validate potential vulnerabilities without impacting the availability of production hosts (while testing in an environment with production data).
  • Look for other types of issues that a commercial vulnerability scanner may not identify (assuming that you will have already performed authenticated vulnerability scans within the boundary).

Networks and hosts outside of the FedRAMP boundary will not be in scope for this internal threat scenario, apart from controls that are inherited or are interconnected, e.g., boundary authentication and authorization mechanisms that leverage your corporate Active Directory (AD) deployment. In this case, the corporate AD infrastructure would be considered in scope and all other corporate assets would remain out of scope.

Attack Vector 3: Tenant to CSP Management System (Web Application)

Attack Type: A full application penetration test that attempts to access your management systems through misconfiguration, a flaw in system design, abuse of intended function, low-code or no-code software deployment, and/or command line interface (CLI). Intended to identify any opportunity that privileged customer accounts would have to compromise your underlying system architecture.

How Does It Work: You would provide your Pen Test Team with privileged level accounts to applications within the production environment, which they would then use to facilitate and identify scenarios where the attacker may go from unauthenticated access to authenticated access to privileged level access. You must provide the highest level of permissions available to customers to conduct this kind of test.

While you might prefer to evaluate a tenant within your development/test environments, these are rarely identical to the production deployment, and so they cannot be used as a valid representation for the FedRAMP penetration test vectors.

Attack Vector 4: Tenant-to-Tenant (Web Application)

Attack Type: A full application test where your Pen Test Team will attempt to use provisional access one of your customers might have to compromise another client.

How Does It Work: You will need to provide two full production customer tenants and their granted access methods that mirror those used by your customers. Your environment must also be set up to test all aspects, including authentication, data access, user permissions, and sessions.

As an authenticated user of the application, the Pen Test Team will focus on gaining access to the other tenant’s data, attempting vertical and horizontal privilege escalation. They’ll also seek to identify and exploit vulnerabilities in the application that could potentially lead to gaining access to the other tenant.

Attack Vector 5: Mobile Application to Target System (iOS and Android)

Attack Type: Simulation of a mobile application user attempting to access your target system or your target system’s mobile application.

How Does It Work: Your Pen Test Team will assess the in-scope Android and/or iOS application(s) and review how it/they handle(s) authorization, as well as cache functionality, data storage, encryption, logging, and other functionality to identify potential local vulnerabilities that could lead to a breach in your environment.

(If your app contains SSL pinning or root/jailbreak detection, please provide builds that have these protections disabled to expedite testing.)

Attack Vector 6: Client-Side Application and/or Agents to Target System

Attack Type: Investigation to ensure any client-side applications do not introduce any new vulnerabilities to the host operating system (OS) nor do they insecurely store sensitive data locally.

How Does It Work: You’ll list any in-scope client-side applications, i.e., components installed locally within a customer environment. These must be included in your FedRAMP authorization boundary and tested. The Pen Test Team will download and install those applications/components on a Virtual Machine (VM)—snapshots of the VM will be made before and after the install. In addition, any traffic will be proxied and reviewed.

• If these applications are essential for your customer's use to interact with the environment or application, they must be included in your authorization boundary and tested as part of your system boundary security assessment. These may include (though not exclusively):
  • Appliances
  • Browser extensions
  • Thick clients
  • Agents
• However, if they’re optional use, these may be included in your tested authorization boundary if your customers and you agree they should be.

Next Steps for Your FedRAMP Pen Test

All of these will need to be performed as part of your FedRAMP preparation moving forward.

However, if a specific attack vector cannot be performed, it will be noted in the SAR as a deviation from the Penetration Testing Guidance. Moreover, understand that a 3PAO might see non-conformance to testing a particular attack vector as a High Risk finding in the SAR Risk Exposure Table (RET).

If you feel that testing the attack vector would negatively impact your production system, you can submit a non-conformance justification for why it cannot be tested to an Authorizing Official (AO), but that may result in delaying your FedRAMP authorization since the AO will need to understand and agree to the deviation or non-conformance.