Commit Graph

1849 Commits

Author SHA1 Message Date
Niels de Vos
582d004ca5 util: store EnvVaultInsecure as string, not bool
The configuration option `EnvVaultInsecure` is expected to be a string,
not a boolean. By converting the bool back to a string (after
verification), it is now possible to skip the certificate validation
check by setting `vaultCAVerify: false` in the Vault configuration.

Fixes: #1852
Reported-by: Bryon Nevis <bryon.nevis@intel.com>
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-02-02 11:24:28 +00:00
Niels de Vos
df81022349 rbd: add support for VAULT_SKIP_VERIFY in KMS ConfigMap
When the KMS VaultTokens is configured through a Kubernetens ConfigMap,
the `VAULT_SKIP_VERIFY` option was not taken into account. The option
maps to the `vaultCAVerify` value in the configuration file, but has the
reverse meaning.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-02-01 13:09:54 +00:00
Mudit Agarwal
d480eb4bda cephfs: ignore BytesQuota field in case it is not set.
This can happen when the subvolume is in snapshot-retained state.
We should not return error for such case as it is a valid situation.

Signed-off-by: Mudit Agarwal <muagarwa@redhat.com>
2021-02-01 09:20:53 +00:00
Madhu Rajanna
584a43dc2c rbd: fix issue in ENV variable check
Currently cephcsi is returning an error
if the ENV variable is set, but it should not.
This commit fixes the the POD_NAMESPACE env
variable issue and as well as the KMS_CONFIG_NAME
ENV variable.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-01-29 10:00:12 +00:00
Niels de Vos
0b7521162c cleanup: rewrite ifElseChains to switch statements
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-01-27 13:03:56 +00:00
Matt Brown
123a26abb3 deploy, helm: enable secret watch in rbac
enables secret ''watch'' rbac permission for ceph-csi-rbd-provisioner role. Fixes 1841.

Signed-off-by: Matt Brown <matthewbrown18@gmail.com>
2021-01-26 15:41:11 +00:00
liulei3
94fc931d44 doc: remove redundant word
Signed-off-by: liulei3 <liulei3@360.cn>
2021-01-25 09:57:12 +00:00
Prasanna Kumar Kalever
0d5efe9147 doc: add more example formats to run e2e tests
Added more example to run e2e and functional tests using `go test` and
`make` commands.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-01-20 12:29:59 +00:00
Flemming Frandsen
47e12a6b6c doc: explain why certain features are unsupported
Signed-off-by: Flemming Frandsen <dren.dk@gmail.com>
2021-01-15 13:10:38 +00:00
Mudit Agarwal
0339df2b02 rebase: update golang version to v1.15.5
To fix math/big CVE, this version of golang has the fix
for this vulnerability.

Signed-off-by: Mudit Agarwal <muagarwa@redhat.com>
2021-01-11 10:05:59 +00:00
Ilya Dryomov
04644c1d58 rbd: enable mapping and unmapping from a network namespace
Make rbdplugin pod work in a non-initial network namespace (i.e. with
"hostNetwork: false") by skipping waiting for udev events when mapping
and unmapping images.  CSI use case is very simple: all that is needed
is a device node which is immediately fed to mkfs, so we should be able
to tolerate udev not being finished with the device just fine.

Fixes: #1323
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2021-01-07 15:34:05 +00:00
Ilya Dryomov
c2493686b7 rbd: introduce appendDeviceTypeAndOptions()
Factor out --device-type and --options formatting.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2021-01-07 15:34:05 +00:00
Ilya Dryomov
d3f31187fc rbd: rename ndbType parameter
Fix "ndb" typo.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2021-01-07 15:34:05 +00:00
Ilya Dryomov
5631b83dd0 rbd: rename mapOptions and options argument slices
With the new support for passing --options, referring to ExecCommand()
argument slices as mapOptions and options is confusing.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2021-01-07 15:34:05 +00:00
Seena Fallah
1f17d25407 e2e: add filesystem PVC resize test for radosNamespace
Signed-off-by: Seena Fallah <seenafallah@gmail.com>
2021-01-05 11:26:09 +00:00
Seena Fallah
fdec9f65b8 rbd: fix namespace json parser for xbdDeviceInfo
rbd device list --format=json returns namespace as a namespace not radosNamespace

Signed-off-by: Seena Fallah <seenafallah@gmail.com>
2021-01-05 11:26:09 +00:00
Yati Padia
995879d349 cephfs: use enum to check resize is supported or not
Currently, we are using bool pointer to find out
the ceph cluster supports resize or not. This
commit replaces the bool pointer with enum.

Signed-off-by: Yati Padia <ypadia@redhat.com>

Fixes#1764
2021-01-04 04:42:58 +00:00
Niels de Vos
61a16012b7 build: only use --cpuset options when the cgroup controller is available
Fixes: #1670
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-23 02:53:51 +00:00
Niels de Vos
90966597b4 ci: require ci/centos/mini-e2e/k8s-1.20 for merging
k8s-1.20 CI jobs seem stable, so there is nothing preventing us from
requiring v1.20 for test results. v1.18 jobs will be removed from the CI
environment, so that version will not be required anymore.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-18 12:05:10 +05:30
Madhu Rajanna
9c7176dbb4 rbd: update mount packges in import path
mount packges is moved from
k8s.io/utils/mount to a new repository
k8s.io/mount-utils. updated code to use
the same.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-17 16:04:54 +00:00
Madhu Rajanna
1db3f03856 e2e: IsRetryableAPIError is not available in kubernetes 1.20.0
IsRetryableAPIError is not available in latest
kubernetes release ie 1.20.0 created a internal
function called isRetryableAPIError for the same.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-17 16:04:54 +00:00
Madhu Rajanna
83559144b1 rebase: update kubernetes to v1.20.0
updated kubernetes packages to latest
release.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-17 16:04:54 +00:00
Niels de Vos
4abe128bd8 e2e: log events from deployment namespace
When tests run and something goes wrong during deployment, not all
information is available. Logging the events from the namespace where
Ceph-CSI (and Vault) is deployed, might help with troubleshooting.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-17 11:40:57 +00:00
Niels de Vos
dd29c6c06b deploy: allow rbd nodeplugin to read ConfigMaps from Tenants
Tenants can have their own ConfigMap that contains connection parameters
to the Vault Service where the PV encyption keys are located. It is
possible for a Tenant to use a different Vault Service than the one
configured by the Storage Admin who deployed Ceph-CSI.

For this, the node-plugin needs to be able to read the ConfigMap from
the Tenants namespace.

See-also: docs/design/proposals/encryption-with-vault-tokens.md
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-16 16:45:29 +00:00
Madhu Rajanna
b3fbcb9c95 rbd: read configuration from the configmap
if the kms encryption configmap is not mounted
as a volume to the CSI pods, add the code to
read the configuration from the kubernetes. Later
the code to fetch the configmap will be moved to
the new sidecar which is will talk to respective
CO to fetch the encryption configurations.

The k8s configmap uses the standard vault spefic
names to add the configurations. this will be converted
back to the CSI configurations.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-16 15:05:05 +00:00
Niels de Vos
e4b16a5c72 util: allow tenants to (re)configure VaultTokens settings
A tenant can place a ConfigMap in their Kubernetes Namespace with
configuration options that differ from the global (by the Storage Admin
set) values.

The ConfigMap needs to be located in the Tenants namespace, as described
in the documentation

See-also: docs/design/proposals/encryption-with-vault-tokens.md
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-16 13:42:52 +00:00
Madhu Rajanna
81061e9f68 util: add support for vault certificates
Added a option to pass the client certificate
and the client certificate key for the vault token
based encryption.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-16 11:01:15 +00:00
Yug
3109160fa0 deploy: Fix snapshot controller deployment
Replace image from canary to the version mentioned
in build.env

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-12-16 07:17:03 +00:00
Niels de Vos
db15458d16 cleanup: use constant for "vault" KMS-type
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-14 14:45:09 +00:00
Niels de Vos
6e2d16ed1e e2e: add test for VaultTokensKMS support
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-14 14:45:09 +00:00
Niels de Vos
cc5684dbd8 util: add support for Hashicorp Vault with Tokens per Tenant
Tenants (Kubernetes Namespaces) can use their own Vault Token to manage
the encryption keys for PVCs. The working is documented in #1743.

See-also: #1743
Closes: #1500
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-14 14:45:09 +00:00
Niels de Vos
cb1899b8c0 deploy: allow rbd nodeplugin to read Secrets from Tenants
In order to fetch the Kubernetes Secret with the Vault Token for a
Tenant, the ClusterRole needs to allow reading Secrets from all
Kubernetes Namespaces (each Tenant has their own Namespace).

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-14 14:45:09 +00:00
Niels de Vos
648f9ccf31 util: support vaultBackendPath and vaultTLSServerName options
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-14 14:45:09 +00:00
Niels de Vos
8f91c672d4 util: add EncryptionKMS.Destroy()
Add a new method to the EncryptionKMS interface so that resources can be
freed when EncryptionKMS instances get freed.

With the move to using the libopenstorage API, a temporary file needs to
store the optional CA certificate. The Destroy() method of the
vaultConnection type now removes this file.

The rbdVolume uses the EncryptionKMS type now, so call the new Destroy()
method from withing rbdVolume.Destroy().

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-14 14:45:09 +00:00
Niels de Vos
eb1ef69cfb util: allow updating settings of vaultConnection
Make it possible to calle initConnection() multiple times. This enables
the VaultTokensKMS type to override global settings with options from a
per-tenant configuration.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-14 14:45:09 +00:00
Niels de Vos
f08182e2fc rbd: pass Owner to GetKMS()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-14 14:45:09 +00:00
Niels de Vos
5bbab25a9f util: move Secrets to vaultConnection
The Secrets is the main object to connect to Vault. This should be part
of the vaultConnection type.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-14 14:45:09 +00:00
Niels de Vos
d28a5a5f23 util: pass map[string]interface{} to initialize KMS
This makes it possible to pass a more complex configuration to the
initialize functions for KMS's. The upcoming VaultTokensKMS can use
overrides for configiration options on a per tenant basis. Without this
change, it would not be possible to consume the JSON configuration file.

See-also: #1743
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-14 14:45:09 +00:00
Niels de Vos
43fa1cddb7 util: use helper function to parse Vault configuration
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-14 14:45:09 +00:00
Niels de Vos
b8fec4df64 doc: fix links for example yaml files
The yaml files for RBD encryption are located in examples/kms/vault, and
not in the examples/rbd directory.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-14 11:14:50 +00:00
Niels de Vos
db40c06e84 doc: add usage for Vault Tokens KMS support
In addition to the Vault KMS support (uses Kubernetes ServiceAccount),
there is the new Vault Tokens KMS feature.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-14 11:14:50 +00:00
Niels de Vos
24a17094a2 doc: multi-tenancy with Vault tokens
Design for adding a new KMS type "VaultTokens" that can be used to
configure a Hashicorp Vault service where each tenant has their own
personal token to manage encryptions keys for PVCs.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-10 15:39:47 +00:00
Madhu Rajanna
0d9fcbc21b cephfs: remove unused cr Credentials
removed unused cr variable.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-10 10:13:55 +00:00
Madhu Rajanna
9a96370942 cephfs: implement purgeVolume with go-ceph
moved frm  ceph fs CLI to go-ceph for
purgeVolume.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-10 10:13:55 +00:00
Madhu Rajanna
34d0ff0d70 cephfs: make purgeVolume method of volumeOptions
converted purgeVolume from function to method
of volumeOptions.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-10 10:13:55 +00:00
Madhu Rajanna
eeec1213cb rebase: update go-ceph to v0.7.0
updating go-ceph to latest 0.7.0
release.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-10 10:13:55 +00:00
Madhu Rajanna
97acd47ae9 build: remove ENV_CSI_IMAGE_VERSION from deploy.sh
as we are defining the image version in the
build.env use the same to build images.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-09 17:14:19 +00:00
Madhu Rajanna
cd96b3359b build: use CSI_IMAGE_VERSION in makefile
updated makefile to use CSI_IMAGE_VERSION
from build.env
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-09 17:14:19 +00:00
Madhu Rajanna
e82cd8339c build: add CSI_IMAGE_VERSION to build.env
CSI_IMAGE_VERSION is needed by makefile and
the centos CI to push the image to the minikube vm.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-09 17:14:19 +00:00
Niels de Vos
bccd5cef46 e2e: validate storing of csi.volume.owner attribute
The csi.volume.owner should get stored when the csi-provisioner sidecar
passes additional metadata. This option is now enabled by default, so
the owner (Kubernetes Namespace) of RBD images is expected to be
available.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-09 13:58:48 +00:00