Serverless computing solutions like AWS Fargate are in widespread use now-a-days due to their various advantages like scalability, productivity, cost-effectiveness, and more. With traditional VMs, users have to manage both the applications and the infrastructure. Users also need to provision and manage compute resources and pay for all the VM instances, rather than the actual amount of resources consumed by the application. With serverless platforms, users only need to build and manage their application, they don’t need to worry about managing, scaling, or patching their infrastructure. Users also pay only for the actual resources required to run their applications.
AWS Fargate is one of the most popular serverless platforms today and is now used in nearly one-third of AWS container environments. AWS Fargate allows you to run your containers without any need to manage the underlying EC2 hosts or to worry about scaling issues. However, securing Fargate containers is a big challenge due to the difficulty of monitoring and controlling your workloads. Traditional security solutions including most cloud security solutions are inefficient and ineffective in protecting serverless deployments.
Below we outline some issues that make monitoring and securing AWS Fargate containers a big challenge.
1. No host access: In a Fargate deployment, AWS takes care of managing and patching the VMs. Users don’t have access to underlying hosts. Fargate containers can’t mount host file systems. While this simplifies management and increases productivity, it also makes monitoring and securing your deployments harder.
2. Difficulty of instrumentation: Applications running inside Fargate instances don’t have root privilege. This makes efficient runtime instrumentation of the user applications to monitor the process activities a very challenging task. Two popular approaches for efficient instrumentation are invasive methods like kernel modules and recent less invasive methods like eBPF. However, both of these approaches are infeasible on Fargate without root privilege. Recently, AWS Fargate added support for ptrace. We can use ptrace for instrumentation, it is accurate as it directly intercepts kernel system calls and cannot be bypassed easily. But, it is not very efficient as it requires high overhead OS context switching and it also may not be portable across different architectures. Without privileged access, it is possible to use the LD_PRELOAD environment variable based approach to instrument code to intercept system calls invoked by libraries such as libc and record the calls and their parameters. However it is not very accurate as applications can call kernel functions directly. It is also easier for attackers to outsmart and bypass this type of monitoring by calling system calls directly instead of using libraries. Overall, there is no straightforward approach to efficiently, accurately, and comprehensively monitor your Fargate workloads with Fargate alone.
3. File system monitoring: In Linux based systems, fanotify and inotify are the two widely used approaches for efficient file system monitoring. However, the fanotify system call requires the CAP_SYS_ADMIN capability, which is not available in Fargate instances. Hence, the only feasible way to efficiently monitor file system activities is to use the inotify system call. Unfortunately, the inotify based approach has some drawbacks that limit how much information we can collect to stop the attacks. We can’t obtain the PID of the processes doing the file activities and there are no mechanisms for making access permission decisions using inotify.
4. Network monitoring: Efficient monitoring of network connections and network packets is also challenging in Fargate environments due to lack of support for efficient kernel level tracing mechanisms such as eBPF. This forces security solutions to fall back to lesser efficient approaches to monitor network connections and packets.
5. Deployment model: Lack of privileged access breaks the standard sidecar model of deploying agents to monitor containers on AWS Fargate. Since there is no access to the host, sidecar agent containers can’t monitor other containers running on the Fargate host. There are some alternative approaches of deploying agents on Fargate instances. First, we can automatically download and install the agents in the application containers when the container starts executing its entrypoint code. This approach, however, violates the principle of container immutability. Second, we can modify the dockerfile to add the agents at the CI/CD step. Third, we can keep the agents as a sidecar container which can be mounted as a volume so that other running containers can access the agent and run the agent after container instantiations. The third is the least invasive and requires minimal changes. However, all three approaches have other issues and challenges. One major issue is that each application container on Fargate will run its own agent as compared to one agent monitoring all containers running on one EC2 instance, thus increasing the overhead of monitoring.
6. Difficult to stop attacks: Security solutions use utility programs like iptables or nftables to block attacker traffic and protect the VMs. However, it is not possible to run iptables or nftables on Fargate instances and block attackers, as these tools also require root privilege.
7. Kubernetes support: Lack of access to the nodes and limited access to Kubernetes APIs inside Fargate containers make securing Kubernetes on Fargate a difficult task as some of the security functionalities depend upon these APIs.
As discussed above, there are a lot of technical challenges and issues in monitoring and securing AWS Fargate containers. Due to lack of access to hosts and lack of support for techniques such as eBPF and fanotify, it requires use of other alternative techniques and careful optimizations to build an efficient and effective monitoring system for securing AWS Fargate deployments.
Deepfence is the only solution that visualizes running AWS Fargate workloads, enables runtime vulnerability scanning, and performs network traffic analysis to provide comprehensive layer 7 protection. Learn more and contact us for a demo.
.png)