Hierarchical Namespace Controller (HNC): a look into the future of Kubernetes Multitenancy

Introducing Hierarchical Namespaces

Author: Adrian Ludwin (Google)Safely hosting large numbers of users on a single Kubernetes cluster has always been a troublesome task. One key reason for this is that different organizations use Kubernetes in different ways, and so no one tenancy model is likely to suit everyone. Instead, Kubernete…

KubernetesFriday, August 14, 2020

Hierarchical Namespace Controller (HNC) is the Google initiative to improve the multi-tenant experience in Kubernetes. Up until today, a cluster-level resource (namespaces) organizes Kubernetes objects. Unfortunately, safe host different users into the same cluster require a high degree of automatization knowledge. HNC tries to fill the gap propagating configuration from "parent" namespaces to "child" ones.

Hierarchical namespaces make it easier to share your cluster by making namespaces more powerful. For example, you can create additional namespaces under your team's namespace, even if you don't have cluster-level permission to create namespaces. Then easily apply policies like RBAC and Network Policies across all namespaces in your team (e.g. a set of related microservices).

Source: multi-tenancy sig repository

TL;DR

Link to our study: https://github.com/sighupio/hnc-example-use-cases

Once we tested the latest HNC version (0.5.1), we could see a lot of value from this proposal. Unfortunately, the development has only started recently but it has been talked about in the community for a long time.

HNC is not going to be generally available (GA) this year:

All HNC issues are assigned to an HNC milestone. So far, the following milestones are defined or planned... v1.0: likely late Q1 or early Q2 2021: HNC recommended for production use

Source: kubernetes-sig/multi-tenancy github repository

Getting started

Thanks to the excellent job by the team behind HNC, getting it ready to test and develop it locally using Kind is child's play:

$ kind create cluster
$ kubectl apply -f https://github.com/kubernetes-sigs/multi-tenancy/releases/download/hnc-v0.5.1/hnc-manager.yaml
$ curl -L https://github.com/kubernetes-sigs/multi-tenancy/releases/download/hnc-v0.5.1/kubectl-hns -o ./kubectl-hns
$ chmod +x ./kubectl-hns
$ export PATH=${PWD}:${PATH}

Note for macOS users: Follow the guide available here.

Then you can start testing it:

$ kubectl create ns hnc-parent
namespace/hnc-parent created
$ kubectl create ns hnc-child-1
namespace/hnc-child-1 created
$ kubectl create ns hnc-child-2
namespace/hnc-child-2 created

$ kubectl hns set hnc-child-1 -p hnc-parent
Setting the parent of hnc-child-1 to hnc-parent
Succesfully updated 1 property of the hierarchical configuration of hnc-child-1
$ kubectl hns set hnc-child-2 -p hnc-parent
Setting the parent of hnc-child-2 to hnc-parent
Succesfully updated 1 property of the hierarchical configuration of hnc-child-2

$ kubectl hns tree -A
default
hnc-parent
├── hnc-child-1
└── hnc-child-2
hnc-system
kube-node-lease
kube-public
kube-system
local-path-storage

By default, Role and RoleBinding objects available in parent namespaces will propagate to child namespaces:

$ kubectl create role pod-reader --verb=get --verb=list --verb=watch --resource=pods -n hnc-parent
role.rbac.authorization.k8s.io/pod-reader created

Then, if you look for the same role in hnc-child-{1,2} namespaces:

$ kubectl get role -n hnc-child-1 --show-labels
NAME         CREATED AT             LABELS
pod-reader   2020-08-24T08:13:00Z   hnc.x-k8s.io/inheritedFrom=hnc-parent
$ kubectl get role -n hnc-child-2 --show-labels
NAME         CREATED AT             LABELS
pod-reader   2020-08-24T08:13:00Z   hnc.x-k8s.io/inheritedFrom=hnc-parent

Currently, it is pretty amazing. It enables a lot of use cases we've dreamed of for years.

Use cases

Kubernetes is a core portion of SIGHUP's business. We work in many industries at multiple size companies, and we face challenges that are not easy to solve without some multi-tenancy features in Kubernetes.

We tested HNC with two different use cases during our study.

Namespace self provisioning

Sometimes developers need to create new namespaces for their developments. As Namespace object is a cluster-level resource, a cluster administrator has to grant access to this resource. After giving permissions to create namespaces, you (as cluster administrator) probably want to propagate some default configurations like:

• NetworkPolicies
• Roles and RoleBindings
• ResourceQuotas
• LimitRanges

Otherwise, you will soon end up with a messy cluster. How do you do that in a vanilla Kubernetes Cluster?

HNC enables this use case. You can define your parent namespace with all the objects you need to have in child namespaces and forget about manually propagating objects. In addition to that, developers do not need to have any cluster-level permission. Using HNC, you need to grant permission to manage sub-namespaces in the parent namespace (namespaced scope).

If you want to read further and test this use case, follow our guide.

Application templates

The previous use case is focused on configuration management of namespaces. This use case is a step forward from the previo1us one. Imagine you are the owner of a multi-tier application with the following layers:

• Frontend
• Backend
• DB

Suppose you are involved in the development of a new frontend feature. In that case, you could benefit from having a well-built backend templated to test your changes in a stable and isolated environment.

In this case, you can define your common application components in a parent namespace including:

• DB Deployments and Services
• Backend Deployments and Services
• Frontend Services

IMPORTANT NOTE: Don't place the frontend deployment in the parent namespace; parts you need to modify shouldn't be in the parent namespace.

Then, create a child namespace to get a namespace ready to use. This new namespace is where you have to place the frontend deployment with your modifications/features.

If you want to read further and test this use case, follow our guide.

Conclusions

HNC tries to fill the gap created by the lack of the multi-tenancy features in Kubernetes. The underlying idea is good enough as a starting point; it needs more love and development power as they already have a considerable amount of feedback from the community.

Some features/issues we are actively reporting are:

• Move the HNCConfiguration from being a cluster-level resource to a (parent) namespace scope configuration. Another option could be to develop the namespace scoped one while maintaining both (merging the setting).
• To override child objects if its name matches a new parent object. Imagine you define a new RBAC Role named developer in a child namespace. If you set the same Role in the parent namespace, it should overwrite child ones with the parent role configuration.
• Fix issues while replicating/propagating unique cluster level data/attributes between parent and child namespaces. As an example: Service objects can not be spread because HNC tries to clone every single attribute from the parent to the child, including spec.ClusterIP value which is unique across the cluster.

You can follow how it is developing by joining the multi-tenancy working group at Google.

Finishing

SIGHUP is interested in contributing to HNC because it has a high potential to become a standard in a few years. As discussed during the assessment period, there are a lot of alternatives to create your (opinionated) multi-tenancy platform based on Kubernetes. Still, it has to be standard somehow at some point soon.

Don't forget to visit the SIGHUP site as well as our GitHub organization and the Git repository containing all the material mentioned in this blog post.