↓ Skip to main content
Le Site de François
  1. Index/

Confidential Cluster

Security Confidential-Computing Kubernetes Attestation Tee
Table of Contents

A Confidential Cluster is a Kubernetes cluster designed so that the cloud or infrastructure operator — including hypervisor administrators, datacenter staff, and anyone who can access the underlying hardware — is entirely outside the trusted computing base. It achieves this by running every Kubernetes node, including control plane nodes, as a Confidential VM, and by extending the confidential boundary to cover not just individual workloads but the cluster’s network traffic, persistent storage, and control plane state. The goal is that a workload owner can cryptographically verify the entire cluster before trusting it, and that no privileged party outside the cluster’s own CVMs can read or tamper with workload data, cluster secrets, or etcd contents.

The distinction from CoCo (Confidential Containers) is architectural scope and threat model. CoCo runs individual pods inside micro-CVMs while the Kubernetes worker nodes and control plane remain on conventional (non-confidential) infrastructure — the kubelet and control plane are untrusted, and CoCo compensates with per-pod attestation and policy enforcement via Trustee. A Confidential Cluster takes the opposite approach: the entire cluster infrastructure — every worker node, every control plane node — is inside CVMs, so the kubelet, etcd, and the Kubernetes API server all run within the confidential boundary. This closes the control plane as an attack surface entirely, but requires the cluster to be built from scratch on CVM-capable hardware rather than layered onto an existing cluster. The two approaches address the same adversary (the infrastructure operator) through different architectural means, and can be combined: a Confidential Cluster running CoCo workloads gives both cluster-level and pod-level confidentiality.

The critical operational challenge of a Confidential Cluster is node admission: how does a new node, booting autonomously in a cloud environment, prove to the existing cluster that it is a genuine, unmodified CVM before being allowed to join and receive the encryption keys it needs to access the cluster’s state disk, network, and etcd? The answer is attestation-gated join: the bootstrapping node produces a hardware attestation report (a TDX Quote or SEV-SNP report) and presents it over an attested TLS connection to a join service running on the existing control plane; only if the report matches the expected node image measurements does the join service issue a Kubernetes bootstrap token and the storage encryption key. This means every node in the cluster has been individually attested against a known-good image before being admitted — node images built along the lines of bootc and protected at runtime by composefs naturally provide the stable, reproducible measurements that make those reference values meaningful. The result is “whole cluster” attestation: a workload owner can derive a single hardware-rooted certificate that transitively covers the node image, the cluster topology, and the configuration of every node that was ever admitted to the cluster.

Relevant Red Hat blog posts
#

Related

Confidential Containers (CoCo)

Security Confidential-Computing Kubernetes Attestation Containers Cncf
Confidential Containers (CoCo) is a CNCF sandbox project that lifts hardware confidential computing — TDX, SEV-SNP, Intel SGX, IBM Secure Execution — up to the Kubernetes pod level, providing a unified software layer that abstracts away the underlying TEE technology. Its defining trust model is unusually strict: the Kubernetes control plane, the kubelet, the container runtime, and the cloud operator are all treated as explicitly untrusted. Only the hardware itself and the workload owner’s own supply chain are in scope for trust.

TEE (Trusted Execution Environment)

Security Confidential-Computing Hardware Attestation Tee
A Trusted Execution Environment (TEE) is a hardware-enforced isolated execution context whose confidentiality and integrity are protected by the CPU itself, rather than by software policy. The defining property of a TEE is that its guarantees hold against an adversary with full control of the software stack outside it — the hypervisor, the host operating system, the BIOS firmware, and even a user with physical access to the machine — because the enforcement is implemented in silicon and cannot be overridden by software. Code and data inside a TEE are encrypted in DRAM using a key held within the CPU’s memory controller or on-die security processor, CPU register state is isolated from the host at context switch boundaries, and memory integrity protection prevents the host from replaying, remapping, or aliasing TEE memory pages. The threat model TEEs are designed against is therefore the infrastructure provider: a cloud operator, a data centre staff member, or a co-tenant who controls the hypervisor layer — the party that traditional virtualisation, namespaces, and access control cannot protect against because they depend on a trusted host kernel.

Trustee

Security Confidential-Computing Attestation Coco Tee Secrets
Trustee is the server-side attestation infrastructure for the CoCo (Confidential Containers) project, previously known as CoCo-KBS. It implements the relying-party and verifier roles from the IETF RATS (Remote ATtestation procedureS) architecture: it receives hardware attestation evidence from workloads running inside TEEs, verifies that evidence against known-good reference values, evaluates it against policy, and — if the workload passes — releases the secrets it needs to operate. Trustee runs outside the TEE in a separately trusted environment (a dedicated server, a different confidential VM, or a Kubernetes operator deployment) and is by design not accessible to the untrusted host or hypervisor.