Container Security

Securing Containers at Run-time with a Kubernetes Security Mesh

By Gary Duan

The term “service mesh” has become increasingly popular in recent years and now it seems like the right time to introduce a “security mesh” for a service mesh. Although the idea of a service mesh is not new, its potential hasn’t been realized until container platforms and software implementations have matured. Before I talk about the security mesh, let’s review the service mesh concept.

[On-demand] There is a video on this topic available here.

What is a Service Mesh?

The success of Kubernetes has been a major enabler for the “service mesh” concept to become a reality, as a “sidecar” container is the ideal form-factor for service mesh functions to be placed together with the service itself. Several implementations have reached the stage of maturity that gives confidedence to enterprise users to start experimenting and deploying the technology.

Two key aspects of the service mesh are the data plane and the control plane. For the data plane, there are solutions such as Envoy, NGINX and Linkerd. For the control plane, there are solutions such as Istio and Conduit.

security mesh for service mesh

Diagram from this post

Modern Security Issues

The tools and processes that are used to deliver and deploy applications have been evolving rapidly. The agility, scale and robustness of modern applications have rendered the traditional security solutions obsolete. If a security solution needs to adapt to the modern cloud native environment, perhaps the service mesh can provide some structure and enablement to implement a security mesh. So, what are the key service mesh principles?

Deployed as Close to the Application as Possible

The first design principle of a service mesh can be explained using the End-to-End Principle, which states that the reliability and security capabilities from inspecting the communication channel between two services cannot fully meet the requirements of the services. The implication is that functions such as load balancing, authentication and telemetry should not be deployed “mid-box,” such as in routers and gateways. Instead, these features should co-locate with the services, as close to the service as possible.

The challenge is that to be deployed truly at the origin of the application would require duplication of software logic in every application that follows this principle. There are not only practical implementation issues but issues with keeping such logic updated.

Use a Sidecar for Application Services

The “sidecar” mechanism made practical for large scale deployment by Kubernetes solves this challenge perfectly. While applications remain intact, the sidecar service running in the same network namespace with the application can be dedicated to traffic management and security functions.

We believe a security solution should be deployed as close to the application as possible as well, effectively creating a security mesh for the service mesh. This brings two benefits. The first benefit is that a container platform, such as Kubernetes, can ensure that the entire application life-cycle can be monitored and protected. When the applications scale, the security solution scales together with the application pods. The second benefit is that this allows the security solution to have complete visibility of application behavior. I will talk about this point more a little later.

Separation of Control Plane and Data Plane

Most service mesh implementation contains two modules, a control plane module and a data plane module. In the case of Istio, Istio is the control plane that makes decisions on policy routing and manages service related data among the services and platform. It configures the data plane, which is Envoy, to most importantly forward network traffic, perform health checks, enforce content-based routing rules, conduct traffic management and implement authentication functions.

istio architecture

Istio Architecture


It is easy to see the traces of the Software Defined Network (SDN) concept here. We believe the security solution for this environment should also be software defined and should have a separate control plane and data plane, unlike traditional routers and firewalls. The security data plane is responsible for inspecting network traffic with high throughput and performance and for gathering telemetry data of network and systems. The security control plane, by interacting with the platform and data plane, provides service discovery data, establishes application behavior models, resolves policy-driven security enforcement. This is how a security mesh can be implemented for the service mesh.

Network and Application Awareness

A service mesh is a dedicated infrastructure layer that handles end-to-end communications for services. By terminating the TCP connection at the data plane proxy, various L4~L7 functions can be performed with knowledge of the application protocols used. For example, based on the HTTP request URL and Headers, connections can be routed to different destinations according to policy definitions.

We believe that understanding both networking and applications is crucial to a  run-time security solution in a container environment. Without access to network communication, an attacker is unable to initiate attacks, expand the blast radius, probe for vulnerabilities, and inject malicious code to exfiltrate data from a target. The network is always where the first-layer of defense-in-depth should take place, before such attacks can reach application workloads. On the other hand, any exploit and breach leaves traces in the system, through files they touch, processes they run and system resources they access. If we can identify any deviations from normal application behavior, and combine those findings with network intelligence, we will be able to pinpoint attacks more accurately and detect the early signs of the security breach.

NeuVector as a Security Mesh

Traditionally, security solutions were always labeled as Network Security, Endpoint, or Server Security. These had to be separate products deployed and managed separately because of the separation of technology and infrastructure and the teams who managed them. There were always debates about  which one solution or approach was more effective for protecting users and detecting attacks.

Now, for the first time in the history, we can see the convergence of these different sides. In the cloud-native environment, where application deployment, management, scaling and routing is virtualized away from the infrastructure, a service mesh with a security mesh is now possible.

NeuVector is able to see security events occurring in the network, in the application (container), and on the host. This valuable converged view enables NeuVector customers to detect and predict attacks at more points in the ‘kill chain’ than was possible from a single security solution in the past.

A security mesh can be deployed today with a solution like NeuVector, which integrates with Kubernetes and auto-scales and updates as application deployments change. In the future, as a service mesh is deployed with technologies like Istio, NeuVector will continue to provide security focused capabilities to enhance the service mesh with a converged view of network, application, and host security events.

NeuVector can protect running containers… find out how

About the Author

Gary Duan

Gary is the Co-Founder and CTO of NeuVector. He has over 15 years of experience in networking, security, cloud, and data center software. He was the architect of Fortinet’s award winning DPI product and has managed development teams at Fortinet, Cisco and Altigen. His technology expertise includes IDS/IPS, OpenStack, NSX and orchestration systems. He holds several patents in security and data center technology.