FAQs

Kamaji is an open-source Kubernetes operator that turns any Kubernetes cluster into a management cluster capable of orchestrating many independent tenant clusters. It runs each tenant’s control plane as ordinary pods inside the management cluster rather than on dedicated machines, which is the pattern known as hosted control planes.

No. Kamaji is an operator that provides managed Kubernetes clusters as a service. It uses unmodified upstream kube-apiserver, kube-scheduler, and kube-controller-manager binaries and relies on kubeadm, so the clusters it produces are conformant upstream Kubernetes.

Kamaji is named after Kamaji, the boiler room operator in the Japanese film Spirited Away, who efficiently manages the bathhouse’s water system.

The TenantControlPlane is a namespaced resource that defines a tenant’s control plane. The DataStore resource defines the persistence backend. When used with Cluster API, the KamajiControlPlane resource represents the control plane in a Cluster API workflow.

The tenant does. Tenant worker nodes are real virtual machines or bare metal machines that belong to the tenant, each belonging to exactly one tenant cluster and running only that tenant’s workloads. They can sit on bare metal, private cloud VMs, public cloud instances, edge locations, or a hybrid of these.

No. Kamaji’s design principle is unidirectional management: communication is strictly one-way, and tenant clusters have no access to and no awareness of the management cluster. There is also no communication between different tenant clusters.

Yes, and this is an explicit design goal. Kamaji decouples the control plane from the infrastructure, so the management cluster can sit on a different infrastructure or cloud provider from the tenant worker machines as long as network reachability is assured.

In EKS, GKE, or AKS the provider owns and operates the control plane as a closed proprietary service, and you manage worker nodes only, with no ability to customize or port the control plane. Kamaji implements the same architectural pattern as open source software you run yourself, which is why it is used as the foundation for independent managed Kubernetes offerings.

etcd, which is the default and the most widely used. MySQL and PostgreSQL-compatible databases through the kine integration. NATS, which is explicitly labeled experimental.

That is configurable. By default Kamaji can persist all tenant clusters in a single datastore, but you can create pools of datastores and assign clusters based on resource, performance, or policy requirements. Each TenantControlPlane can name a specific datastore, and when none is named the operator default is used. Rackspace, for example, runs one MySQL database per control plane rather than a shared etcd.

NATS support is experimental because it does not support multi-tenancy. A NATS-backed datastore can host exactly one tenant control plane, and a second control plane referencing the same datastore will fail to reconcile.

Yes, by patching the datastore reference. During the migration the control plane is placed in read-only mode by an admission webhook and its status moves from Ready to Migrating and back to Ready. The default migration timeout is five minutes and is adjustable by annotation. After migration the kubelet must be restarted on all tenant worker nodes, and data left behind on the source datastore must be removed manually. Live migration is only supported between datastores using the same driver.

Yes, kamaji-etcd, a Helm chart that deploys a multi-tenant etcd as a three-replica StatefulSet. It is installed by the Kamaji Helm chart as the default datastore.

The documentation describes hundreds or thousands of isolated clusters. Rackspace states publicly that a management cluster can host hundreds of control planes, and Clastix has referenced adopter deployments in the range of 500 or more clusters on premises.

The reference benchmark runs 100 tenant control planes against a single shared etcd datastore with AWS EKS as the management cluster. Full reconciliation of all 100 completed in approximately seven and a half minutes with all control planes reaching Ready. Averaged out, that is one control plane reconciled every 4.5 seconds.. Kamaji CPU usage fluctuated during certificate generation peaks and memory peaked around 600 MiB. A larger multi-datastore benchmark is described as future work. Other performance figures that circulate in marketing material are not from this benchmark and should be attributed rather than asserted.

A main manager reconciles administrative resources, and a separate manager is spun up per tenant control plane to run the controllers for that tenant’s addons and kubeadm resources.

Through a Kubernetes Service. LoadBalancer is the default, and NodePort and ClusterIP are also supported. The default API port is 6443 and is configurable.

Konnectivity solves control plane to worker node communication where direct connectivity is not possible. A Konnectivity server runs as a sidecar inside each tenant control plane pod, and agents on the worker nodes open outbound connections to it, creating tunnels used for exec, logs, port-forward, metrics, and health collection. It is enabled by default and can be disabled. Because workers dial outbound, this is what allows worker nodes to sit across regions, availability zones, or even different clouds behind firewalls and NAT.

Either as a DaemonSet running on every node, or as a Deployment with a configurable replica count for a lower footprint.

Automatically. Recent Konnectivity releases align with the API server version and Kamaji selects the matching one. For very recent Kubernetes releases the corresponding Konnectivity images may not yet be published upstream, in which case an earlier available version can be pinned, since a version skew is permitted.

Patch the Kubernetes version field on the TenantControlPlane (or KamajiControlPlane, if running in Cluster API mode) to a value permitted by the Kubernetes version skew policy.

Blue/green. All new control plane pods are created at once while the existing pods keep serving until the new ones are ready, which is fast and minimizes downtime. The trade-offs are that it temporarily requires double the control plane pod resources and that starting all new pods simultaneously can put load on the datastore connection.

Yes. For resource-constrained environments, or to protect the datastore from load spikes, the deployment strategy can be set to a rolling update with a bounded surge and unavailability.

No. Kamaji does not provide helpers for tenant worker nodes. You upgrade them with kubeadm, or use the Cluster API control plane provider for declarative lifecycle management.

The API server certificate, the API server kubelet client certificate, the datastore certificate, the front proxy client certificate, and the Konnectivity certificate when enabled, plus the client certificates inside the admin, controller-manager, scheduler, and Konnectivity kubeconfigs.

Yes. Certificates use kubeadm defaults, so they are valid for one year, and a certificate lifecycle controller inspects them and triggers rotation automatically, by default the day before expiry. The deadline is configurable, for example to rotate a week ahead. Manual rotation is available by annotating the relevant Secrets, individually or in bulk by label selector.

The CA uses the kubeadm default validity of ten years and is rotated manually by annotation, at which point the control plane enters a rotating state. It is deliberately manual because a new CA requires restarting all components and all nodes and redistributing the new CA and node certificates.

Kamaji generates the etcd client certificate itself. For MySQL, PostgreSQL, and NATS the PKI is offloaded externally: the controller deletes the referenced Secret to trigger rotation and an external PKI supplies the renewed certificates.

Because tenant control planes are ordinary pods in the management cluster, standard Kubernetes backup tooling applies. The documented approach uses Velero to back up the tenant’s namespace and restore the TenantControlPlane and its Secrets, after which the control plane cycles through NotReady back to Ready. Velero is used as an infrastructure-agnostic backup strategy, primarily to serve customers without a vendor-backed backup solution.

No. The documented procedure restores the control plane resource only. If the datastore itself has been lost it must be restored first, using the backup and restore strategy belonging to that datastore technology. Datastore backup is out of scope for Kamaji itself, and in practice the backup design is tailored per environment as part of a services engagement.

They complement each other. Kamaji simplifies control plane management while Cluster API handles infrastructure abstraction and lifecycle. Kamaji can be used standalone, with nodes joined by Ansible, Terraform, or kubeadm scripts, or installed as a Cluster API control plane provider.

There are dedicated documentation pages for vSphere, Proxmox VE, and OpenStack, plus a general page for other providers. The control plane provider repository documents integration with AWS, Azure, Equinix, Hetzner, KubeVirt, Metal3, metal-stack, Nutanix, OpenStack, OUTSCALE, Tinkerbell, vSphere, IONOS Cloud, and Proxmox by IONOS Cloud.

Yes, provided they use kubeadm as the bootstrap provider. Migrations are handled as part of a professional services engagement.

Yes. Communication runs through a secure gRPC tunnel via Konnectivity, so packet inspection rules need to account for that. OpenStack customers typically use Octavia-powered load balancers, and Clastix has also developed a pure Layer 3 network plugin based on OVS localport that facilitates API server to worker node communication securely.

Yes. A separate CRD Helm chart allows custom resource definition lifecycle to be managed independently of the operator chart, which suits Flux and Argo CD workflows. The documented pattern also runs Flux in the management cluster and points it at each tenant’s admin kubeconfig Secret, so addons can be reconciled into tenant clusters centrally without installing Flux in every tenant.

Yes, the Kamaji Console, a web UI installed by Helm into the management cluster. It lists all tenant control planes with their name, namespace, status, endpoint, version, and datastore. Additional capabilities such as infrastructure driver management, application delivery, centralized authentication, auditing, monitoring, and backup are unlocked through the Clastix enterprise offering.

Prometheus-format metrics on the controller manager metrics endpoint, including counts and status for tenant control planes, datastores, and certificates, handler timings, and build information, alongside the usual Go runtime and controller-runtime metrics. A ready-made Grafana dashboard ships in the repository, and tenant control plane component metrics can be scraped from the API server, controller manager, and scheduler ports.

No, and this is documented explicitly. By default the manager and kubeconfig generator metrics endpoints are served in plaintext with no authentication and are reachable by any workload with network access to the pod. Enable secure serving to use HTTPS with bearer token authentication, or restrict access with a NetworkPolicy.

Yes, by default. A telemetry component periodically reports the Kamaji version, Kubernetes version, tenant control plane versions, and control plane counts by status, along with a UUID, to a Clastix-managed backend. Clastix states that it does not collect personal information. Telemetry can be disabled through a Helm value or a controller flag.

Apache 2.0. The same applies to the Cluster API control plane provider and to kamaji-etcd. The code is provided as-is with no warranties.

Clastix states that all tenant clusters built with Kamaji are CNCF conformant, on the basis that they use upstream unmodified binaries and kubeadm. The documented way to verify this yourself is to run Sonobuoy in certified conformance mode against a tenant cluster, and Clastix maintains a dedicated conformance repository.

No. Kamaji is hosted in the Clastix GitHub organization under Apache 2.0. Capsule is the Clastix-created project that sits in the CNCF, at Sandbox maturity. The two claims are frequently conflated: Kamaji produces CNCF-conformant clusters, but Kamaji is not a CNCF-hosted project.

There are three artifact types. Latest is built from every commit on the main branch and is not recommended for production because nothing is pinned. Edge is published monthly with a year-month-increment version and is the mainstream open-source channel. Stable semantic-version artifacts have not been published in the open-source project since July 2024; stable container images and Helm charts are available on a subscription basis from Clastix. This is a material consideration for anyone evaluating Kamaji for production on the open-source channel alone.

The published version matrix in the documentation covers recent management and tenant cluster versions, but it trails the release notes, which have announced support for newer Kubernetes minor versions. Confirm the current supported range against the latest release notes or with Clastix before committing to a version.

Separate closed-source components available only with an active subscription, delivered as images and Helm charts pushed to the customer’s own OCI repository. The one currently documented is the Ingress Addon, which exposes tenant control planes behind an ingress controller rather than consuming one load balancer per cluster, which conserves IPv4 addresses. HAProxy Kubernetes Ingress is the supported controller, and subscribers can request others.

Yes. Clastix offers subscription-based, enterprise-grade support plans for Kamaji, from the people who wrote it.

OVHcloud uses it in its Managed Kubernetes Service Standard offering, confirmed in OVHcloud’s own corporate press release and in a CNCF case study, which describes a fully open-source stack including Kamaji, Cluster API, Cilium, and Argo CD. Rackspace runs it inside Rackspace Spot, using Cluster API for lifecycle and one MySQL database per control plane. NVIDIA uses Kamaji in the DOCA Platform Framework to provision control planes for BlueField DPU clusters, documented in NVIDIA’s own product documentation and referenced in a Red Hat Developer article. Cozystack, a CNCF Sandbox project, is built on Kamaji. The project’s adopters file additionally lists Mistral AI, Platform9, Aruba Cloud, Namecheap, Tinext Cloud, UNICC, Sovereign Cloud Stack, Netsons, WOBCOM, and others, with Orange, TIM, and IONOS Cloud recorded under research and development.

Jeremie Monsinjon, Head of Containers at OVHcloud: “Kamaji works exactly as expected: it’s simple, efficient, scalable, and I especially appreciate how Clastix has always been available for technical discussions and support.” Kevin Carter, Director at Rackspace: “We are running the open-source project Kamaji within our Rackspace Spot platform today, and the results are impressive.”

They isolate at different layers. A vCluster runs inside a namespace of a host cluster and has no real worker nodes of its own: a syncer copies pods down to the host cluster, which schedules them on shared host nodes; in recent versions, VCluster mimicked what Kamaji offered, such as the ability to let real worker nodes join the Virtual Cluster Control Plane, but there’s no infrastructure integration via Cluster API. Kamaji provides a real control plane with real, tenant-owned worker nodes, so isolation extends to the infrastructure rather than stopping at the API, and with a open and flexible integration with Cluster API as cluster lifecycle orchestrator. Performance comparisons published by either vendor should be treated as marketing, since no neutral head-to-head benchmark exists.

All four implement hosted control planes. k0smotron is tied to the k0s distribution and also acts as a bootstrap and infrastructure provider, whereas Kamaji is distribution-neutral upstream Kubernetes and deliberately control-plane-only. Red Hat’s HyperShift, which powers OpenShift hosted control planes, is OpenShift-only on both the management and hosted sides, while Kamaji is vanilla Kubernetes and vendor-neutral on the node side. SAP’s Gardener uses the same pattern, which it calls kubeception, but is a complete end-to-end Kubernetes-as-a-service platform with a much broader scope and a heavier footprint, while Kamaji is a narrower component designed to compose with Cluster API. kcp is sometimes mentioned in the same breath but is a different category: it serves Kubernetes-like APIs with no pods and no worker nodes at all.

They solve different multi-tenancy problems and complement each other. Capsule gives multiple tenants isolated namespaces inside a single shared cluster. Kamaji gives each tenant a dedicated control plane, which is what you need when tenants require full cluster-admin permissions or their own Kubernetes version.