Kubernetes has become ubiquitous in the deployment of applications and microservices. Its ability to take the concept of containerization to gargantuan scale has been a major driver for adoption. However with such capabilities, comes complexities and if not configured appropriately, can result in vulnerable configurations and a potential for compromise. Let’s dive in.
The Obvious One: The Patched
When conducting penetration testing against DevOps environments, one of the first steps is to discover hosts, enumerate services and identify vulnerabilities. From our experience with assessing “unmanaged” Kubernetes deployments, where the DevOps team deploys on hardware or IaaS infrastructure, we see a common trend of the environment not being kept up to date.
The first reason to keep Kubernetes up to date is features. Prior to version 1.8, Kubernetes lacked a Role Based Access Control (RBAC) solution which severely limited administrators ability to control who does what within the environment. The second reason and arguably the most important reason to keep Kubernetes up to date is security patches. In the first 8 months of 2021, there have been 16 Common Vulnerability and Exposures notices (CVEs) published for Kubernetes which included privilege escalation and remote code execution level vulnerabilities.
Authentication Material: KubeConfig
Administrators of Kubernetes perform most of their tasks through the Kubeadm application which interacts with Kube API. This application authenticates users via the Kubeconfig file (~/.kube/config). This file is very sensitive and if found can allow an attacker administrative access to the Kubernetes Cluster.
During penetration testing, testers will commonly search for this file rather than trying to run exploits against the Kubernetes. In cases where a developer’s laptop was compromised by phishing atttack, the first thing a penetration tester will do is search the laptop for the Kubeconfig file, along with all the other gold that usually sits in the user’s ~/Desktop, ~/Downloads and OneNote Notebooks.
Occasionally we find sensitive files like Kubeconfig in code repositories accidentally committed and improperly removed.
Most Common Attack Surface: The Kube API
When targeting Kubenetes, the API will be the majority of the attack surface. The API is the central nervous system for all the components within the Kubernetes infrastructure, and gaining access to it can result in a total compromise of the cluster.
RBAC capabilities is a major security control within Kubernetes and in particular with the Kube API. Out-of-box Kubernetes RBAC defaults are insufficient and overly permissive likely leading to the potential to launch hostile pods, access Etcd’s metadata service and access Kubernetes secrets store.
To note, additional attack surfaces do exist, particularly in unmanaged environments where the Master/Worker Node underlying host operating system is exposed.
Namespace and Network Segmentation
Namespaces are used to separate pods and provide access control to the pods through RBAC mechanisms. However, Namespaces are not isolation layers like traditional hypervised virtualization and should not be considered as such. That said, Enforcing RBAC to explicitly allow access to specific Namespace via Kubeconfig and Kubelet certificates is the best way to limit access and minimize the blast radius of compromised credentials.
Kubernetes provides software define networking capabilities and similar to Namespaces, can provide separation capabilities at the network layer. However, by default the network is not segmented allowing a penetration testers the ability to pivot from Namespace to Namespace and likely beyond the cluster’s software-defined network and out and on to the LAN. In the spirit of limiting the blast radius, networks should be segmented and connectivity between pods and the rest of the network should be controlled.
When Trusting Third-Party Solutions Goes Wrong
Whether you are using a CI Server such as Jenkins or an application deployment solution such as Helm-Tiller, one must consider the potential for attack. Many times these tools are given Kubeconfig files to authenticate and interact with Kubernetes with substantial “write permissions”. These tools, like most open source software can become vulnerable or misconfigured over time and if not kept up to date and hardened can be an additional attack surface the Kubernetes administrators are not considering.
COMMENTS