It seems that the version of the key/value engine can not always be
detected for Hashicorp Vault. In certain cases, it is required to
configure the `VAULT_BACKEND` (or `vaultBackend`) option so that a
successful connection to the service can be made.
The `kv-v2` is the current default for development deployments of
Hashicorp Vault (what we use for automated testing). Production
deployments default to version 1 for now.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
This change resolves a typo for installing the CSIDriver
resource in Kubernetes clusters before 1.18,
where the apiVersion is incorrect.
See also:
https://kubernetes-csi.github.io/docs/csi-driver-object.html
[ndevos: replace v1betav1 in examples with v1beta1]
Signed-off-by: Thomas Kooi <t.j.kooi@avisi.nl>
nlreturn linter requires a new line before return
and branch statements except when the return is alone
inside a statement group (such as an if statement) to
increase code clarity. This commit addresses such issues.
Updates: #1586
Signed-off-by: Rakshith R <rar@redhat.com>
Sometimes there are failures in the e2e suite when connecting to the
etcdserver fails. The following error was caught:
INFO: Error getting pvc "rbd-pvc" in namespace "rbd-1318": Get "https://192.168.39.222:8443/api/v1/namespaces/rbd-1318/persistentvolumeclaims/rbd-pvc": dial tcp 192.168.39.222:8443: connect: connection refused
FAIL: failed to create PVC with error failed to get pvc: Get "https://192.168.39.222:8443/api/v1/namespaces/rbd-1318/persistentvolumeclaims/rbd-pvc": dial tcp 192.168.39.222:8443: connect: connection refused
If etcdserver was only briefly unavailable, one or more retries might be
sufficient to have the test pass.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Since kubernetes 1.16 clone ( create a new volume from exisiing volume)
functionality is at Beta state and kubernetes v1.18, the snapshot functionality
has been lifted to GA. Ceph CSI drivers have been supporting this feature
for last few releases and users are heavily using this feature since then.
We also have good amount of e2e test case which cover volume creation
from PVC backend or iow, PVC as a datasource. With that, this PR
proposes of lifting this feature support to Beta with v3.4.0 version.
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
Since kubernetes 1.17 snapshot functionality is at Beta state
and external snapshotter 3.0.3. Since v4.0.0 of snapshotter controller
and kubernetes v1.20, the snapshot functionality has been lifted to GA.
Ceph CSI drivers have been supporting this feature for last few releases
and users are heavily using this feature since then.
We also have good amount of e2e test case which cover volume creation
from snapshot backend or iow, snapshot as a datasource. With that, this PR
proposes of lifting this feature support to Beta with v3.4.0 version.
Updates# https://github.com/ceph/ceph-csi/issues/2199
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
Since kubernetes 1.17 snapshot functionality is at Beta state
and external snapshotter 3.0.3. Since v4.0.0 of snapshotter controller
and kubernetes v1.20, the snapshot functionality has been lifted to GA.
Ceph CSI drivers have been supporting this feature for last few releases
and users are heavily using this feature since then.
We also have good amount of e2e test case which cover volume creation
from snapshot backend or iow, snapshot as a datasource. With that, this PR
proposes of lifting this feature support to Beta with v3.4.0 version.
Updates# https://github.com/ceph/ceph-csi/issues/2199
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
we have volume PV metrics support available in our driver along
with the grpc metrics support (EnableGRPCMetrics) we added in between
for csi operations. The latter is getting deprecated and the current
mention in the support matrix on metrics support confuse many.
This PR explictly mention this support in the docs to volume/PV metrics
This PR also add a seperate row for block mode PV metrics
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
In addition to the single ServiceAccount KMS support for Hashicorp
Vault, Ceph-CSI can now use a ServiceAccount per Tenant as well. This
adds the user-documentation with references to the example deployment
files.
Closes: #2222
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Sometimes it happens that the deployment of Hashicorp Vault fails.
Deployment is one of the 1st steps that are done when starting the e2e
suite, and the Kubernetes cluster may still be a little overloaded while
it is settling down. It should be possible to retry and succeed after a
while.
Fixes: #2288
Signed-off-by: Niels de Vos <ndevos@redhat.com>
This commit resolves errorlint issues
which checks for the code that will cause
problems with the error wrapping scheme.
Updates: #1586
Signed-off-by: Yati Padia <ypadia@redhat.com>
revive linter checks for var-declaration
format.
For example:
"e2e/rbd_helper.go:441:36: var-declaration:
should drop = nil from declaration of
var noPVCValidation; it is the zero value (revive)
var noPVCValidation validateFunc = nil"
Updates: #1586
Signed-off-by: Yati Padia <ypadia@redhat.com>
snapshot controller make sure the pvc which is the source for the
snapshot request wont get deleted while snapshot is getting created,
so we dont need to check for any ongoing delete operation here on the
volume.
Subjected code path in snapshot controller:
```
pvc, err := ctrl.getClaimFromVolumeSnapshot(snapshot)
.
..
pvcClone.ObjectMeta.Finalizers = append(pvcClone.ObjectMeta.Finalizers, utils.PVCFinalizer)
_, err = ctrl.client.CoreV1().PersistentVolumeClaims(pvcClone.Namespace).Update(..)
```
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
warnings from golangci-lint:
e2e/pod.go:207:122: directive `//nolint:unparam,lll // cn can be used
with different inputs later` is unused for linter unparam (nolintlint)
func execCommandInContainer(f *framework.Framework, c, ns, cn string,
opt *metav1.ListOptions) (string, string, error) { //nolint:unparam,lll
// cn can be used with different inputs later
e2e/pod.go:307:70: directive `//nolint:unparam // skipNotFound can be
used with different inputs later` is unused for linter unparam (nolintlint)
func deletePodWithLabel(label, ns string, skipNotFound bool) error {
//nolint:unparam // skipNotFound can be used with different inputs later
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
Now that the healer functionaity for mounter processes is available,
lets start, using it.
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
Problem:
-------
For rbd nbd userspace mounter backends, after a restart of the nodeplugin
all the mounts will start seeing IO errors. This is because, for rbd-nbd
backends there will be a userspace mount daemon running per volume, post
restart of the nodeplugin pod, there is no way to restore the daemons
back to life.
Solution:
--------
The volume healer is a one-time activity that is triggered at the startup
time of the rbd nodeplugin. It navigates through the list of volume
attachments on the node and acts accordingly.
For now, it is limited to nbd type storage only, but it is flexible and
can be extended in the future for other backend types as needed.
From a few feets above:
This solves a severe problem for nbd backed csi volumes. The healer while
going through the list of volume attachments on the node, if finds the
volume is in attached state and is of type nbd, then it will attempt to
fix the rbd-nbd volumes by sending a NodeStageVolume request with the
required volume attributes like secrets, device name, image attributes,
and etc.. which will finally help start the required rbd-nbd daemons in
the nodeplugin csi-rbdplugin container. This will allow reattaching the
backend images with the right nbd device, thus allowing the applications
to perform IO without any interruptions even after a nodeplugin restart.
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
Nodeplugin needs below cluster roles:
persistentvolumes: get
volumeattachments: list, get
These additional permissions are needed by the volume healer. Volume healer
aims at fixing the volume health issues at the very startup time of the
nodeplugin. As part of its operations, volume healer has to run through
the list of volume attachments and understand details about each
persistentvolume.
The later commits will use these additional cluster roles.
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
As part of stage transaction if the mounter is of type nbd, then capture
device path after a successful rbd-nbd map.
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
clone controller make sure there is no delete operation happens
on the source PVC which has been referred as the datasource of
clone PVC, we are safe to operate without looking at delete
operation lock in this case.
Subjected code in the controller:
...
if claim.Spec.DataSource != nil && rc.clone {
err = p.setCloneFinalizer(ctx, claim)
...
}
if !checkFinalizer(claim, pvcCloneFinalizer) {
claim.Finalizers = append(claim.Finalizers, pvcCloneFinalizer)
_, err := p.client.CoreV1().PersistentVolumeClaims(claim.Namespace).Update(..claim..)
}
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
Considering kubelet make sure the stage and publish operations
are serialized, we dont need any extra locking in nodePublish
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
Considering kubelet make sure the stage and publish operations
are serialized, we dont need any extra locking in nodePublish
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
Considering kubelet make sure the unstage and unpublish operations
are serialized, we dont need any extra locking in nodeUnpublish
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
Considering kubelet make sure the unstage and unpublish operations
are serialized, we dont need any extra locking in nodeUnpublish
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
Minikube has bumped it's support for latest Kubernetes version to
1.22.0-beta.0. This might improve our CI jobs with Kubernetes 1.22 too.
See-also: https://github.com/kubernetes/minikube/releases/tag/v1.22.0
Signed-off-by: Niels de Vos <ndevos@redhat.com>
When an error occurs, the pvc object is overwritten in the
PollImmediate() loop. Re-using the pvc.Namespace results in error
messages like
Error getting pvc in namespace: '': an empty namespace may not be set when a resource name is provided
and prevents the retry by PollImmediate() to never succeed. Storing the
namespace in a local variable prevents this from happening.
Reported-by: Rakshith R <rar@redhat.com>
Signed-off-by: Niels de Vos <ndevos@redhat.com>
There are regular CI failures where etcdserver times out. These errors
seem not to get caught by any of the existing error comparing. Matching
the error by string should prevent temporary etcdserver issues now too.
Updates: #2218Closes: #1969
Signed-off-by: Niels de Vos <ndevos@redhat.com>
parseTenantConfig() only allowed configuring a defined set of options,
and KMSs were not able to re-use the implementation. Now, the function
parses the ConfigMap from the Tenants Namespace and returns a map with
options that the KMS supports.
The map that parseTenantConfig() returns can be inspected by the KMS,
and applied to the vaultTenantConnection type by calling parseConfig().
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Kubelet sometimes reports the following error:
failed to "StartContainer" for "vault-init-job" with CreateContainerConfigError: container has runAsNonRoot and image will run as root
Setting securityContext.runAsUser resolves this.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
The ServiceAccount "ceph-csi-vault-sa" is expected to be placed in the
Namespace "tenant" so that the provisioner and node-plugin fetch the
ServiceAccount from a Namespace where Ceph-CSI is not deployed.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
The provisioner and node-plugin have the capability to connect to
Hashicorp Vault with a ServiceAccount from the Namespace where the PVC
is created. This requires permissions to read the contents of the
ServiceAccount from an other Namespace than where Ceph-CSI is deployed.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
This new KMS uses a Kubernetes ServiceAccount from a Tenant (Namespace)
to connect to Hashicorp Vault. The provisioner and node-plugin will
check for the configured ServiceAccount and use the token that is
located in one of the linked Secrets. Subsequently the Vault connection
is configured to use the Kubernetes token from the Tenant.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
This makes the Tenant configuration for Hashicorp Vault KMS connections
more modular. Additional KMS implementations that use Hashicorp Vault
with per-Tenant options can re-use the new vaultTenantConnection.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
A new KMS that supports Hashicorp Vault with the Kubernetes Auth backend
and ServiceAccounts per Tenant (Kubernetes Namespace).
Updates: #2222
Signed-off-by: Niels de Vos <ndevos@redhat.com>
This commit resolves parallel test issues
and also excludes internal/util/conn_pool_test.go
as those test can't run in parallel.
Updates: #1586
Signed-off-by: Yati Padia <ypadia@redhat.com>
This commit resolves godot linter issue
which says "Comment should end in a period (godot)".
Updates: #1586
Signed-off-by: Yati Padia <ypadia@redhat.com>
This adds a new `kmsConfig` interface that can be used to validate
different KMS services and setting. It makes checking for the available
support easier, and fetching the passphrase simpler.
The basicKMS mirrors the current validation of the KMS implementations
that use secrets and metadata. vaultKMS can be used to validate the
passphrase stored in a Vault service.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
