Commit Graph

118 Commits

Author SHA1 Message Date
Niels de Vos
aa52afff09 cleanup: move SecretsKMS in own file
Prepared for an enhanced API to communicate with a KMS and keep the DEK
storage separate. The crypto.go file is already mixed with different
functions, so moving the KMS part into its own file, just like we have
for Hashicorp Vault KMS's.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-03-12 10:11:47 +00:00
Niels de Vos
6c8bc79771 ci: add unit tests for SecretsKMS
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-03-04 12:11:07 +00:00
Niels de Vos
74d218df8d rbd: disable rbd_discard_on_zeroed_write_same for thick-allocation
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-02-19 11:55:40 +00:00
Niels de Vos
dc81e001cf cleanup: remove unused MissingPassphrase error type
Storing a passphrase is now done while the volume is created. There is
no need to (re)generate a passphrase when it can not be found.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-02-17 17:51:13 +00:00
Niels de Vos
9b6c2117f3 rbd: set encryption passphrase on CreateVolume
Have the provisioner create the passphrase for the volume, instead of
doign it lazily at the time the volume is used for the 1st time. This
prevents potential races where pods on different nodes try to store
different passphrases at the (almost) same time.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-02-17 17:51:13 +00:00
Niels de Vos
a42c4b5855 util: convert VAULT_SKIP_VERIFY to "vaultCAVerify" KMS option
"VAULT_SKIP_VERIFY" is a standard Hashicorp Vault environment variable
(a string) that needs to get converted to the "vaultCAVerify"
configuration option in the Ceph-CSI format.

The value of "VAULT_SKIP_VERIFY" means the reverse of "vaultCAVerify",
this part was missing in the original conversion too.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-02-16 13:05:47 +00:00
Madhu Rajanna
dd6ce7b441 rbd: fix error check when reading vaultCAFromSecret
check correct error variable when reading
vaultCAFromSecret.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-02-04 14:58:40 +00:00
Madhu Rajanna
e9782d86ad rbd: fix incorrect reading of client cert key
fix incorrect reading of client cert key.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-02-04 14:58:40 +00:00
Madhu Rajanna
f63ccb0cce rbd: store VaultCAVerify as a string
storing VaultCAVerify as a string.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-02-04 14:58:40 +00:00
Madhu Rajanna
bf5c36822f rbd: set tenant in kms object
the tenant/namespace is needed to read the certificates,
this commit sets the tenant in kms object.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-02-04 14:58:40 +00:00
Madhu Rajanna
22ae4a0b16 rbd: change key in secret for cert and tls
currently, the keys for kms certificates/keys in a
secret is ca.cert, tls.cert and
tls.key, this commit changes the key from ca.cert
and tls.cert to cert and tls.key to key.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-02-04 14:58:40 +00:00
Madhu Rajanna
b370d9afb6 rbd: unmarshal the data read from file
if are reading the kms data from the file.
than only we need to unmarshal. If we are reading
from the configmap it already returns the unmarshal
data.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-02-04 14:58:40 +00:00
Niels de Vos
582d004ca5 util: store EnvVaultInsecure as string, not bool
The configuration option `EnvVaultInsecure` is expected to be a string,
not a boolean. By converting the bool back to a string (after
verification), it is now possible to skip the certificate validation
check by setting `vaultCAVerify: false` in the Vault configuration.

Fixes: #1852
Reported-by: Bryon Nevis <bryon.nevis@intel.com>
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-02-02 11:24:28 +00:00
Niels de Vos
df81022349 rbd: add support for VAULT_SKIP_VERIFY in KMS ConfigMap
When the KMS VaultTokens is configured through a Kubernetens ConfigMap,
the `VAULT_SKIP_VERIFY` option was not taken into account. The option
maps to the `vaultCAVerify` value in the configuration file, but has the
reverse meaning.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-02-01 13:09:54 +00:00
Madhu Rajanna
584a43dc2c rbd: fix issue in ENV variable check
Currently cephcsi is returning an error
if the ENV variable is set, but it should not.
This commit fixes the the POD_NAMESPACE env
variable issue and as well as the KMS_CONFIG_NAME
ENV variable.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-01-29 10:00:12 +00:00
Madhu Rajanna
b3fbcb9c95 rbd: read configuration from the configmap
if the kms encryption configmap is not mounted
as a volume to the CSI pods, add the code to
read the configuration from the kubernetes. Later
the code to fetch the configmap will be moved to
the new sidecar which is will talk to respective
CO to fetch the encryption configurations.

The k8s configmap uses the standard vault spefic
names to add the configurations. this will be converted
back to the CSI configurations.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-16 15:05:05 +00:00
Niels de Vos
e4b16a5c72 util: allow tenants to (re)configure VaultTokens settings
A tenant can place a ConfigMap in their Kubernetes Namespace with
configuration options that differ from the global (by the Storage Admin
set) values.

The ConfigMap needs to be located in the Tenants namespace, as described
in the documentation

See-also: docs/design/proposals/encryption-with-vault-tokens.md
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-16 13:42:52 +00:00
Madhu Rajanna
81061e9f68 util: add support for vault certificates
Added a option to pass the client certificate
and the client certificate key for the vault token
based encryption.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-16 11:01:15 +00:00
Niels de Vos
db15458d16 cleanup: use constant for "vault" KMS-type
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-14 14:45:09 +00:00
Niels de Vos
cc5684dbd8 util: add support for Hashicorp Vault with Tokens per Tenant
Tenants (Kubernetes Namespaces) can use their own Vault Token to manage
the encryption keys for PVCs. The working is documented in #1743.

See-also: #1743
Closes: #1500
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-14 14:45:09 +00:00
Niels de Vos
648f9ccf31 util: support vaultBackendPath and vaultTLSServerName options
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-14 14:45:09 +00:00
Niels de Vos
8f91c672d4 util: add EncryptionKMS.Destroy()
Add a new method to the EncryptionKMS interface so that resources can be
freed when EncryptionKMS instances get freed.

With the move to using the libopenstorage API, a temporary file needs to
store the optional CA certificate. The Destroy() method of the
vaultConnection type now removes this file.

The rbdVolume uses the EncryptionKMS type now, so call the new Destroy()
method from withing rbdVolume.Destroy().

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-14 14:45:09 +00:00
Niels de Vos
eb1ef69cfb util: allow updating settings of vaultConnection
Make it possible to calle initConnection() multiple times. This enables
the VaultTokensKMS type to override global settings with options from a
per-tenant configuration.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-14 14:45:09 +00:00
Niels de Vos
f08182e2fc rbd: pass Owner to GetKMS()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-14 14:45:09 +00:00
Niels de Vos
5bbab25a9f util: move Secrets to vaultConnection
The Secrets is the main object to connect to Vault. This should be part
of the vaultConnection type.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-14 14:45:09 +00:00
Niels de Vos
d28a5a5f23 util: pass map[string]interface{} to initialize KMS
This makes it possible to pass a more complex configuration to the
initialize functions for KMS's. The upcoming VaultTokensKMS can use
overrides for configiration options on a per tenant basis. Without this
change, it would not be possible to consume the JSON configuration file.

See-also: #1743
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-14 14:45:09 +00:00
Niels de Vos
43fa1cddb7 util: use helper function to parse Vault configuration
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-14 14:45:09 +00:00
Niels de Vos
9160a5309e cleanup: standardize error format in VaultKMS.GetPassphrase()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-09 08:35:35 +00:00
Niels de Vos
e3ea8ca0b1 cleanup: standardize error format in util.GetTopologyFromDomainLabels()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-09 08:35:35 +00:00
Niels de Vos
6d5de7458d cleanup: standardize error format in util.k8sGetNodeLabels()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-09 08:35:35 +00:00
Niels de Vos
752841d213 cleanup: standardize error format in util.readClusterInfo()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-09 08:35:35 +00:00
Niels de Vos
3dfe65d17c cleanup: return error type in GetCryptoPassphrase()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-09 08:35:35 +00:00
Niels de Vos
c8c8176a37 cleanup: return error type in util.storeKey()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-09 08:35:35 +00:00
Niels de Vos
5565a902ac rebase: split vaultConnection from VaultKMS
In order to re-use the configuration of Vault, split a new
vaultConnection type from the VaultKMS type.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-30 19:16:39 +00:00
Madhu Rajanna
8ebb9a1ba0 cleanup: fix misspell words
fixed misspell words detected by  codespell

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-11-30 08:46:48 +01:00
Madhu Rajanna
39b1f2b4d3 cleanup: fix mispell words
fixed mispell words in the repo.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-11-29 12:47:46 +05:30
Niels de Vos
7824cb5ed7 util: use %w when forwarding an error
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-29 04:03:59 +00:00
Niels de Vos
5fba89f783 cleanup: use libopenstorage/secrets for Vault access
Instead of the hand-rolled Vault usage, use the libopenstorage/secrets
package that provides a nice API. The support for Vault becomes much
simpler and maintainable that way.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-29 04:03:59 +00:00
Madhu Rajanna
30af703a2f rbd: add controller to main
initialize and start the rbd controller when
we the driver type is controller.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-11-28 18:50:00 +00:00
Madhu Rajanna
8d3a44d0c4 rbd: add minsnapshotsonimage flag
An rbd image can have a maximum number of
snapshots defined by maxsnapshotsonimage
On the limit is reached the cephcsi will
start flattening the older snapshots and
returns the ABORT error message, The Request
comes after this as to wait till all the
images are flattened (this will increase the
PVC creation time.  Instead of waiting till
the maximum snapshots on an RBD image, we can
have a soft limit, once the limit reached
cephcsi will start flattening the task to
break the chain. With this PVC  creation time
will only be affected when the hard limit
(minsnapshotsonimage) reached.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-11-18 05:59:20 +00:00
Humble Chirammal
992d4b92bb util: NewCredentials() dont have any callers
We have below exported function in credentials.go which is not
called from anywhere in the repo. Removing it for the same reason.

```
 // NewCredentials generates new credentials when id and key
 // are provided.

 func NewCredentials(id, key string) (*Credentials, error) {
...
```

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-10-21 21:04:11 +00:00
Humble Chirammal
25617929f0 rbd: use different variable instead of builtin cap function
`cap` builtin function returns the capacity of a type. Its not
good practice to use this builtin function for other variable
names, removing it here

Ref# https://golang.org/pkg/builtin/#cap

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-10-21 12:30:18 +00:00
Niels de Vos
fe62348cfb util: add ClusterConnection.GetFSAdmin()
The go-ceph CephFS Admin API needs a rados.Conn to connect the Admin
interface to the Ceph cluster. As the rados.Conn is internal to the
ClusterConnection type, add GetFSAdmin() to create a new FSAdmin
instance for a specific connection.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-14 15:16:56 +00:00
Humble Chirammal
36ee8b2240 rbd: make sure csi rbd pv encryption defaulting to luks2 version
Fixes: #https://github.com/ceph/ceph-csi/issues/1564

As per release notes this new version is compatible with previous
version of LUKS.
https://www.saout.de/pipermail/dm-crypt/2017-December/005771.html

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-10-13 05:11:25 +00:00
Madhu Rajanna
d816df4f42 util: add helper to validate ROX request
added a helper function to validate is create
request for ROX is for volume from source.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-09-10 13:47:23 +00:00
Madhu Rajanna
eeb0859f99 util: rename FatalLog to FatalLogMsg for parity
rename FatalLog to FatalLogMsg to keep functions
in parity functions ends with Msg if its only message
logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
3d12456493 util: add WarningLog to log with context
added WarningLog to log with context

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
b223924044 util: rename WarningLog to WarningLogMsg
renamed WarningLog function to WarningLogMsg
to log with message.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
8c32045c4e util: use ErrorLog for context based logging
use ErrorLog for context based logging for
more detailed logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
7c24f4d597 util: add ErrorLog for log with context
added a new function ErrorLog to log with
context.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00