AWS Container Threats – How to Detect Threats in the Public Cloud
By Fei Huang
Today more and more applications are running in a public cloud in containers. A common question we hear is “Do AWS container threats exist and how can we detect them?” For the application administrator or a security team, it is always interesting to know what and how their containers look like from security point of view.
The recent ransomware issues with MongoDB and Elasticsearch are not the only AWS container threats to worry about. It is important to have visibility into how containers are working and the details of their network connections.
We did some simple tests on AWS using a test application and captured some real-world threat examples to show how containers are just as vulnerable as any other application infrastructure. We deployed several application containers in an AWS ECS test environment. We used the standard configuration and turned on platform security features such as security groups. Only one publicly facing port was opened for each of application. For example, port 1080 was opened for an NGINX container which served as a load balancer for the application’s HTTP web requests. Then we deployed the NeuVector security containers, set the system in protect mode, and left the application running for a few days. This is the automatically generated network map and application segmentation created.
We noticed that every two or three days a few alerts were reported. Examining the violation logs clearly showed some suspicious connections from an external network. These violations were also shown as being blocked (red line) in the NeuVector network map.
After taking a closer look at the NeuVector logs we determined that these are port scans from various places in Asia and the US. This is a very low frequency port scan which an anomaly-based firewall won’t be able to detect. And gateway security devices will be hard-pressed to catch them because of the high noise and false positive levels. It’s difficult for three or four HTTP tries per day to stand out from all the other alerts normally coming in. Existing blacklist solutions with IP address based protection may not be up to date as well because the attack sources are changing every day. However, with a layered security strategy, some threats can be caught at the entry point into the application containers, as shown in the logs below.
A quick analysis shows that someone was connecting to this NGINX container. The probe detects port status then tries different application protocols like HTTP.
If there’s no network-based container security blocking these requests, the NGINX container will reply with the standard “400 bad request” including meta-data about the server. The attacker could easily find out this is NGINX and it was on version v1.11.1!
The next steps would be straightforward. First find out the known vulnerabilities of NGINX v1.11.1, then build some real attacks against it. For example DoS, Code execution, Buffer overflow or privileges escalation. Once the attackers gain access to this container with certain privileges, the applications, containers and internal networks become wide open to be exploited from internal sources.
Most companies have not implemented east-west security protections which monitor and secure internal traffic. But with containers, the volume and potential threats from east-west traffic increase tremendously. We’ve actually seen real cases of attacks like this and the victim companies have lost a ton of money because of these attacks.
A further analysis based on the NeuVector logs shows that this attacker is not only doing port scanning but also trying to look for open and vulnerable proxy servers. If there are any misconfigured servers discovered successfully this could become a stepping stone for further attacks.
In our tests we have the NeuVector solution protecting the NGINX application containers. Even though the port scans connect to the open port 1080, NeuVector determines in real-time that this is not legitimate HTTP traffic. The suspicious HTTP connections were then blocked in real-time and the port scanning gets rejected before reaching the NGINX container at all. NeuVector is able to block these attempts early and cause the port scanner to fail.
NeuVector will alert and block on any suspicious connection attempts regardless if it’s from an external network or internal network. Both north-south and east-west traffic are protected at the application entry point at run-time. NeuVector also has the capability to capture the network payload for deep analysis and forensic purposes.
Detailed logs can be exported to an existing gateway and platform security solution as well, so the attacker’s 2nd try could be blocked at the front door of the entire virtual environment if perimeter security is integrated with NeuVector.
Network security is still a critical requirement in a containerized environment. Containers make it more difficult to get visibility and security especially in public cloud environments. Port scanning is a simple but often used technique for attackers to identify network services running on a host and exploit vulnerabilities. Although this post talks about AWS container threats these threats exist for any public and private cloud container deployments. The AWS container environment is no less or more secure by default than other environments.
With more and more services being migrated to micro-services and containers, monolithic large applications will become hundreds of small containers. The more containers that are running, the more network interfaces that will be introduced, and the more network attack surfaces which are exposed to become port scanning targets.
The risk is extremely high if the scanning was initialized from an internal hijacked or trusted node. In these cases gateway security and platform security are likely to be bypassed. A solution like NeuVector which is powered by virtualization and automation technologies will be able to provide strong and continuous protection for containerized applications.