From ed298341a65e01dbe2229e6b28dc2013f106aaea Mon Sep 17 00:00:00 2001 From: Niels de Vos Date: Wed, 30 Jun 2021 09:02:58 +0200 Subject: [PATCH] doc: proposal for KMS with ServiceAccount per Tenant A new KMS that supports Hashicorp Vault with the Kubernetes Auth backend and ServiceAccounts per Tenant (Kubernetes Namespace). Updates: #2222 Signed-off-by: Niels de Vos --- .../proposals/encryption-with-vault-sa.md | 83 +++++++++++++++++++ 1 file changed, 83 insertions(+) create mode 100644 docs/design/proposals/encryption-with-vault-sa.md diff --git a/docs/design/proposals/encryption-with-vault-sa.md b/docs/design/proposals/encryption-with-vault-sa.md new file mode 100644 index 000000000..733a0b31f --- /dev/null +++ b/docs/design/proposals/encryption-with-vault-sa.md @@ -0,0 +1,83 @@ +# Encrypted Volumes with a ServiceAccount per Tenant + +## Requirements + +- Hashicorp Vault with Kubernetes Auth +- each Tenant (=Namespace) has a ServiceAccount to connect to Vault +- each Tenant can override Vault configuration options (like `BackendPath`) + +## Similarities with available KMS implementations + +Ceph-CSI provides several KMS implementations with different functionalities. +Two implementations use Hashicorp Vault and could be used as a base for a new +KMS implementation. Or, if changes would be minimal, a configuration option to +one of the implementations can be added. + +Different KMS implementations and their configurable options can be found at +[`csi-kms-connection-details.yaml`](../../../examples/kms/vault/csi-kms-connection-details.yaml). + +### VaultTokensKMS + +- global configuration in the Namespace where Ceph-CSI is deployed +- optional configuration per Tenant, allows for overriding Vault options +- fetches a Secret from the Tenants (volume owner) namespace + +An example of the per Tenant configuration options are in +[`tenant-config.yaml`](../../../examples/kms/vault/tenant-config.yaml) and +[`tenant-token.yaml`](../../../examples/kms/vault/tenant-token.yaml). + +Implementation is in [`vault_tokens.go`](../../../internal/util/vault_tokens.go). + +### Vault + +- uses Kubernetes Auth with single service account (aka storage admin account) + +Implementation is in [`vault.go`](../../../internal/util/vault.go). + +## Extension or New KMS implementation + +Normally ServiceAccounts are provided by Kubernetes in the containers +filesystem. This only allows a single ServiceAccount and is static for the +lifetime of the Pod. Ceph-CSI runs in the namespace of the storage +administrator, and has access to the single ServiceAccount linked in the +deployments of the Pods. + +For Ceph-CSI to dynamically use ServiceAccounts from tenants, the following +steps need to be taken: + +- use the Namespace of the volume to identify the Tenant +- identify the right ServiceAccount in the Tenants Namespace +- read and parse the ServiceAccount contents +- provide the ServiceAccount contents as a (temporary) directory structure +- setup the Vault connection to use the contents from the ServiceAccount to + replace the default (`AuthKubernetesTokenPath: + /var/run/secrets/kubernetes.io/serviceaccount/token`) + +Currently the Ceph-CSI components may read Secrets and ConfigMaps from the +Tenants namespace. These permissions need to be extended to allow Ceph-CSI to +read the contents of the ServiceAccount(s) in the Tenants namespace. + +## High-Level Design + +### Global Configuration + +1. a StorageClass links to a KMS configuration by providing the `kmsID` parameter +1. a ConfigMap in the namespace of the Ceph-CSI deployment contains the KMS + configuration for the `kmsID` + ([`csi-kms-connection-details.yaml`](../../../examples/kms/vault/csi-kms-connection-details.yaml)) + +When Ceph-CSI receives a `CreateVolume` request, the parameters from the +StorageClass and details about the requested Volume are available. The Ceph-CSI +provisioner checks the `kmsID` parameter and fetches the matching KMS +configuration from the ConfigMap. + +### Tenant Configuration + +1. needs ServiceAccount with a known name with permissions to connect to Vault +1. optional ConfigMap with options for Vault that override default settings + +A `CreateVolume` request contains the owner (Namespace) of the Volume. +The KMS configuration indicates that additional attributes need to be fetched +from the Tenants namespace, so the provisioner will fetch these. The additional +configuration and ServiceAccount are merged in the provisioners configuration +for the KMS-implementation while creating the volume.