ceph-csi/docs/design/proposals/encryption-with-vault-sa.md

Encrypted Volumes with a ServiceAccount per Tenant

Requirements

• Hashicorp Vault with Kubernetes Auth
• each Tenant (=Namespace) has a ServiceAccount to connect to Vault
• each Tenant can override Vault configuration options (like BackendPath)

Similarities with available KMS implementations

Ceph-CSI provides several KMS implementations with different functionalities. Two implementations use Hashicorp Vault and could be used as a base for a new KMS implementation. Or, if changes would be minimal, a configuration option to one of the implementations can be added.

Different KMS implementations and their configurable options can be found at csi-kms-connection-details.yaml.

VaultTokensKMS

• global configuration in the Namespace where Ceph-CSI is deployed
• optional configuration per Tenant, allows for overriding Vault options
• fetches a Secret from the Tenants (volume owner) namespace

An example of the per Tenant configuration options are in tenant-config.yaml and tenant-token.yaml.

Implementation is in vault_tokens.go.

Vault

• uses Kubernetes Auth with single service account (aka storage admin account)

Implementation is in vault.go.

Extension or New KMS implementation

Normally ServiceAccounts are provided by Kubernetes in the containers filesystem. This only allows a single ServiceAccount and is static for the lifetime of the Pod. Ceph-CSI runs in the namespace of the storage administrator, and has access to the single ServiceAccount linked in the deployments of the Pods.

For Ceph-CSI to dynamically use ServiceAccounts from tenants, the following steps need to be taken:

• use the Namespace of the volume to identify the Tenant
• identify the right ServiceAccount in the Tenants Namespace
• read and parse the ServiceAccount contents
• provide the ServiceAccount contents as a (temporary) directory structure
• setup the Vault connection to use the contents from the ServiceAccount to replace the default (AuthKubernetesTokenPath: /var/run/secrets/kubernetes.io/serviceaccount/token)

Currently the Ceph-CSI components may read Secrets and ConfigMaps from the Tenants namespace. These permissions need to be extended to allow Ceph-CSI to read the contents of the ServiceAccount(s) in the Tenants namespace.

High-Level Design

Global Configuration

1. a StorageClass links to a KMS configuration by providing the kmsID parameter
2. a ConfigMap in the namespace of the Ceph-CSI deployment contains the KMS configuration for the kmsID (csi-kms-connection-details.yaml)

When Ceph-CSI receives a CreateVolume request, the parameters from the StorageClass and details about the requested Volume are available. The Ceph-CSI provisioner checks the kmsID parameter and fetches the matching KMS configuration from the ConfigMap.

Tenant Configuration

1. needs ServiceAccount with a known name with permissions to connect to Vault
2. optional ConfigMap with options for Vault that override default settings

A CreateVolume request contains the owner (Namespace) of the Volume. The KMS configuration indicates that additional attributes need to be fetched from the Tenants namespace, so the provisioner will fetch these. The additional configuration and ServiceAccount are merged in the provisioners configuration for the KMS-implementation while creating the volume.