The huge advances in ‘Shift Left’ processes makes it possible to deliver code to production that is secure and largely free from vulnerable dependencies. Among other things, these processes typically involve matching dependencies against public vulnerability lists from Mitre, Red Hat, Debian, and other projects.
Vulnerability lists don’t stand still. CVEs are published through the NVD at a rate of about 50 per day, so the risk of a new vulnerability being found in production is significant. Furthermore, third-party production components may not be scanned in a way that is as rigorous or up-to-date as you would like.
The Log4j vulnerability, disclosed in December 2021, is an excellent case in point. As with all zero-day exploits, this was published without advanced warning, leaving enterprises scrambling to locate running software that contained the vulnerable Java component. (If you'd like to learn more, check out our blog post on how ThreatMapper helps enterprises find and fix their most exploitable log4j2 vulnerabilities.)
It quickly became clear that applications created by the enterprise were not the only area of concern; a great many third-party applications and tools also bundled that vulnerable component. ‘Shift Left’ scanning alone is not sufficient to identify supply-chain vulnerabilities.
Deepfence’s open source ThreatMapper project helps you address this gap. ThreatMapper scans your in-production workloads using current vulnerability feeds, and will find vulnerabilities that your ‘Shift Left’ scanning may have missed – there can be 1,000’s! ThreatMapper then reduces the list by assessing the risk-of-exploit, so you know what is most critical to fix first. It considers the attack vector, proximity to the network attack surface, and whether the affected workload has been seen to receive untrusted, external traffic.
ThreatMapper is composed of two components – an on-premises management console, and a series of sensors that you deploy to your production platforms. In this tutorial, we’ll show you how to install and use ThreatMapper in a Kubernetes environment. If you have any questions along the way, head on over to GitHub, or ask any questions you may have in our community Slack channel.
In this tutorial, we’ll go through the following steps:
To install the management console on a Docker host, first grab the Docker compose file:
wget https://github.com/deepfence/ThreatMapper/raw/master/deployment-scripts/docker-compose.yml
Then spin the console up:
docker-compose -f docker-compose.yml up --detach
It typically takes about 60 seconds for those container images to start up and initialize. Take note of the external IP address of the Docker host, as we will need that for the next step.
For a more scalable, persistent deployment, you can run the management console on a Kubernetes Cluster. Check out the Kubernetes installation instructions.
Access the management console using a web browser (use the IP address noted above) and create the first admin account. From this point onwards, new users in your organization must be invited by an admin user; open registration ceases.
Go to the Settings -> User Management page and take a copy of the API key for your installation:
You’ll need the API key and the IP address of the management console for the next step - registering sensors to call back to the management console.
ThreatMapper sensors are installed as privileged containers, and support Kubernetes, Docker and AWS Fargate. ThreatMapper sensors may also be used to monitor and secure bare-metal and virtual machine workloads, where the container is run in a docker runtime in the host.
On Kubernetes, the ThreatMapper sensor is deployed as a DaemonSet, managed using helm:
helm repo add deepfence https://deepfence-helm-charts.s3.amazonaws.com/threatmapper
helm install deepfence-agent deepfence/deepfence-agent \
--set managementConsoleUrl=---CONSOLE-IP--- \
--set deepfenceKey=---DEEPFENCE-API-KEY---
One pod per host is started, and it uses privileged access to standard APIs to monitor network traffic and interrogate running containers.
On Docker, the sensor is installed thus:
docker run -dit --cpus=".2" --name=deepfence-agent --restart on-failure \
--pid=host --net=host --privileged=true \
-v /sys/kernel/debug:/sys/kernel/debug:rw -v /var/log/fenced \
-v /var/run/docker.sock:/var/run/docker.sock -v /:/fenced/mnt/host/:ro \
-e USER_DEFINED_TAGS="" -e MGMT_CONSOLE_URL="---CONSOLE-IP---" \
-e MGMT_CONSOLE_PORT="443" -e DEEPFENCE_KEY="---DEEPFENCE-API-KEY---" \
deepfenceio/deepfence_agent_ce:latest
Initially, the sensors will discover what workloads are running in the platform, and identify network flows between each workload.
Very quickly, you’ll be able to explore your application topology. The ThreatMapper management console builds graph and table-based views that you can use to drill down and explore what workloads are present:
So far, so good, but let’s get to the interesting part – the vulnerability scanning!
The management console needs to bootstrap the vulnerability feeds, which can take anything from 10-45 minutes. Look for a ‘last updated’ message in the Settings -> Diagnosis page for confirmation:
Then select a workload and begin a scan. For workloads in Kubernetes, you can use the ‘Advanced Options’ to scan the entire cluster (nodes and pods). The scan is queued and results are populated in the ‘Vulnerabilities’ report when they become available:
ThreatMapper pulls vulnerability feeds from dozens of different sources. The broadest is the NVD CVE list and ThreatMapper will use that over duplicates in other feeds such as Red Hat or Debian security announcements. ThreatMapper also pulls language-specific feeds for Java, Ruby, NodeJS, Python, and others.
A list of theoretical vulnerabilities is of little practical use. Vulnerabilities of low severity, requiring local access for exploit are much less concerning than high-severity, network-exploitable vulnerabilities that are present in workloads that receive external network traffic.
The “Most Exploitable Vulnerabilities” report provides exactly that information. In addition to the severity, it considers the attack vector, proximity to the network attack surface, and whether the affected workload has been observed to receive untrusted, external traffic:
You can examine the report in detail, and discover the nature of the vulnerabilities that ThreatMapper has prioritized, and possible remedial actions:
In this tutorial we showed you how to install the two ThreatMapper components, the management console and the lightweight sensors. We showed you how to visualize your running services (Topology tab) and your vulnerabilities (Vulnerabilities tab). In your discovered vulnerabilities list, we showed you how to get detailed information about a specific vulnerability, and how to see your most exploitable vulnerabilities, the ones you must fix immediately, based on ThreatMapper’s ability to consider a comprehensive set of runtime conditions to identify how exploitable a vulnerability is within your environment.
Next here are a few steps you can take to get started with ThreatMapper:
.png)
