ceph-csi/docs/design/proposals/encryption-with-vault-sa.md

# Encrypted Volumes with a ServiceAccount per Tenant

## Requirements

- Hashicorp Vault with Kubernetes Auth
- each Tenant (=Namespace) has a ServiceAccount to connect to Vault
- each Tenant can override Vault configuration options (like `BackendPath`)

## Similarities with available KMS implementations

Ceph-CSI provides several KMS implementations with different functionalities.
Two implementations use Hashicorp Vault and could be used as a base for a new
KMS implementation. Or, if changes would be minimal, a configuration option to
one of the implementations can be added.

Different KMS implementations and their configurable options can be found at
[`csi-kms-connection-details.yaml`](../../../examples/kms/vault/csi-kms-connection-details.yaml)
.

### VaultTokensKMS

- global configuration in the Namespace where Ceph-CSI is deployed
- optional configuration per Tenant, allows for overriding Vault options
- fetches a Secret from the Tenants (volume owner) namespace

An example of the per Tenant configuration options are in
[`tenant-config.yaml`](../../../examples/kms/vault/tenant-config.yaml) and
[`tenant-token.yaml`](../../../examples/kms/vault/tenant-token.yaml).

Implementation is in [`vault_tokens.go`](../../../internal/util/vault_tokens.go)
.

### Vault

- uses Kubernetes Auth with single service account (aka storage admin account)

Implementation is in [`vault.go`](../../../internal/util/vault.go).

## Extension or New KMS implementation

Normally ServiceAccounts are provided by Kubernetes in the containers'
filesystem. This only allows a single ServiceAccount and is static for the
lifetime of the Pod. Ceph-CSI runs in the namespace of the storage
administrator, and has access to the single ServiceAccount linked in the
deployments of the Pods.

For Ceph-CSI to dynamically use ServiceAccounts from tenants, the following
steps need to be taken:

- use the Namespace of the volume to identify the Tenant
- identify the right ServiceAccount in the Tenants Namespace
- read and parse the ServiceAccount contents
- provide the ServiceAccount contents as a (temporary) directory structure
- setup the Vault connection to use the contents from the ServiceAccount to
  replace the default (`AuthKubernetesTokenPath:
  /var/run/secrets/kubernetes.io/serviceaccount/token`)

Currently, the Ceph-CSI components may read Secrets and ConfigMaps from the
Tenants namespace. These permissions need to be extended to allow Ceph-CSI to
read the contents of the ServiceAccount(s) in the Tenants namespace.

## High-Level Design

### Global Configuration

1. a StorageClass links to a KMS configuration by providing the `kmsID`
   parameter
1. a ConfigMap in the namespace of the Ceph-CSI deployment contains the KMS
   configuration for the `kmsID`
   ([`csi-kms-connection-details.yaml`](../../../examples/kms/vault/csi-kms-connection-details.yaml))

When Ceph-CSI receives a `CreateVolume` request, the parameters from the
StorageClass and details about the requested Volume are available. The Ceph-CSI
provisioner checks the `kmsID` parameter and fetches the matching KMS
configuration from the ConfigMap.

### Tenant Configuration

1. needs ServiceAccount with a known name with permissions to connect to Vault
1. optional ConfigMap with options for Vault that override default settings

A `CreateVolume` request contains the owner (Namespace) of the Volume. The KMS
configuration indicates that additional attributes need to be fetched from the
Tenants namespace, so the provisioner will fetch these. The additional
configuration and ServiceAccount are merged in the provisioners' configuration
for the KMS-implementation while creating the volume.