Commit Graph

425 Commits

Author SHA1 Message Date
Niels de Vos
dbbda5473b e2e: pass non-empty Namespace/Name in deletePVCAndPV()
When getting the PVC or PV failed, the returned object may contain empty
values. If that happens, a retry uses the empty values for Namespace and
Name, which will never be successful.

Instead, use the Namespace and Name attributes from the original object,
and not from the object returned by the Get() call.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-06-30 00:27:27 +00:00
Niels de Vos
2df55a55a3 e2e: use exclusive-lock together with lock_on_read
When using `lock_on_read`, the RBD image needs to have the
`exclusive-lock` feature enabled too.

Fixes: #3221
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-06-29 17:40:17 +00:00
Prasanna Kumar Kalever
b56511c0c8 e2e: reduce defaultCloneCount to 3
CI is failing very frequently hitting resource leaks issue,
until we solve the root cause for resource leaks reducing the clone
count from 10 to 3.

related: #2327
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2022-06-29 11:08:09 +00:00
Madhu Rajanna
d7bf51b255 rebase: update csi-snapshotter to v6
as we have v6.0.0 of the external snapshotter
updating the dependency for the same.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-06-16 18:45:33 +00:00
Robert Vasek
79cb1b3849 e2e: add test for cephfs snapshot-backed volumes
Signed-off-by: Robert Vasek <robert.vasek@cern.ch>
2022-06-16 09:44:27 +00:00
Madhu Rajanna
4b57cc3ec5 rbd: add support for rbd striping
RBD supports creating rbd images with
object size, stripe unit and stripe count
to support striping. This PR adds the support
for the same.

More details about striping at
https://docs.ceph.com/en/quincy/man/8/rbd/#striping

fixes: #3124

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-06-09 18:59:00 +00:00
Prasanna Kumar Kalever
8364493eb9 e2e: add tests to validate cluster name
Validate with both volumes and snapshots

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2022-06-08 16:23:59 +00:00
Madhu Rajanna
1952a9b4b3 ci: fix all linter errors found in golangci-lint
Fixing all the linter errors found in golang-ci
lint v1.46.2

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-06-03 12:55:54 +00:00
Humble Chirammal
63be1df343 e2e: remove snapshot beta CRD mention from the doc
The snapshot CRD is no longer Beta and this commit adjust the same.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-05-26 13:43:17 +00:00
Humble Chirammal
2cb52d6535 e2e: remove the generic ephemeral volume support check
at present we have the check for kube version 1.21 in the tests
which no longer required as it falls under supported kubernetes
versions with the driver.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-05-26 13:43:17 +00:00
Rakshith R
b2f0d62ad8 e2e: add tests for nfs snapshot and restore
Signed-off-by: Rakshith R <rar@redhat.com>
2022-05-24 18:13:02 +00:00
Rakshith R
707ad086b7 e2e: add pvc-pvc clone e2e test
Signed-off-by: Rakshith R <rar@redhat.com>
2022-05-24 18:13:02 +00:00
Rakshith R
952105e551 e2e: remove TODO update nfs node-plugin with fix csi-driver-nfs#319
This commits removes
TODO: update nfs node-plugin that has kubernetes-csi/csi-driver-nfs#319
Since, the nfsplugin image is already updated to v4.0.0.

Signed-off-by: Rakshith R <rar@redhat.com>
2022-05-17 21:56:46 +00:00
Prasanna Kumar Kalever
ba6052e896 e2e: workaround fix for omap leaks
2 omap objects are getting leaked in the e2e tests, this change is to
workaround them for now.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2022-05-12 18:47:27 +00:00
Prasanna Kumar Kalever
ea8a3548e6 e2e: validate omap objects per cephfs test case
validate omap count in every testcase right after
validateSubvolumeCount()

Fixes: #2834
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2022-05-12 18:47:27 +00:00
Prasanna Kumar Kalever
dcf51a10b1 e2e: validate omap objects per rbd test case
validate omap count in every testcase right after
validateRBDImageCount()

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2022-05-12 18:47:27 +00:00
Prasanna Kumar Kalever
53f96a8b2a e2e: add utility functions
such as:
listCephFSFileSystems()
getCephFSPoolName()
validateOmapCount()
compareStdoutWithCount()

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2022-05-12 18:47:27 +00:00
Prasanna Kumar Kalever
89a7ada12f e2e: add various testcases to validate unset of metadata
Validate that we:
* Unset the PVC metadata on the rbd image created for the snapshot
* snapshot metadata on CreateVolume from snapshot

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2022-05-12 15:54:09 +00:00
Niels de Vos
bfef0279a7 e2e: retry getting Pods before executing commands
On occasion the Pods have not been (re)started before they get listed.
This can result in an empty list. It can occur during RBD testing where
Pods are restarted before `uname` is executed. In case the Pods are not
available yet, the test will fail with the "podlist is empty" error.

By adding a retry when the list of Pods is empty, the tests should
become a little more stable.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-05-12 07:43:34 +00:00
Niels de Vos
9f08cb7ea0 e2e: re-order deployment components
Some of the deployment artifacts refer to others (like ServiceAccount in
a Deployment). If the dependencies are not available (yet), there will
be errors reported in the logs. By deploying the components in a more
correct order, fewer errors are reported, making the logs a little
easier to understand.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-05-11 14:58:28 +00:00
Niels de Vos
9d7a9144f1 e2e: add ExpandVolume test for NFS-volumes
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-05-10 17:43:59 +00:00
Niels de Vos
eab2ec42bf e2e: allow kubectlDelete to succeed with NotFound error
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-05-10 00:43:43 +00:00
Niels de Vos
17fea0459d e2e: add -nfs-driver=... option to select provisioner
When testing NFS-provisioning on a cluster that has an NFS-provisioner
and node-plugins deployed with a different driver-name, it is very
useful to have a commandline option to change the name of the
provisioner that is placed in the StorageClass.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-05-10 00:43:43 +00:00
Niels de Vos
9c91710d68 e2e: run NFS-provisioner tests together with CephFS
NFS testing will automatically be enabled when CephFS is enabled. This
makes sure the NFS tests run in the CI where there are different jobs
for CephFS and RBD. With a dedicated testNFS variable, it is still
possible to only run the NFS tests, when both CephFS and RBD are
disabled.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-05-10 00:43:43 +00:00
Niels de Vos
eb70fb9fd4 e2e: add minimal tests for NFS-provisioner
The tests for the NFS-provisioner can be run by passing -deploy-nfs and
-test-nfs as parameters to the `go test` or `e2e.test` command.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-05-10 00:43:43 +00:00
Niels de Vos
782f08e2f0 ci: execCommandInDaemonsetPod should not return unused stdout
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-05-10 00:43:43 +00:00
Niels de Vos
5c1584671e e2e: no need to set Namespace for StorageClass
StorageClasses are cluster-wide, and do not need to have a Namespace
set.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-05-10 00:43:43 +00:00
Madhu Rajanna
1197b94149 e2e: add getPersistentVolume helper function
added getPersistentVolume helper function
to get the PV and also try if there is any API
error to improve the CI.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-05-06 15:55:54 +00:00
Madhu Rajanna
89d9ec0823 e2e: add getPersistentVolumeClaim helper function
added getPersistentVolumeClaim helper function
to get the PVC and also try if there is any API
error to improve the CI.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-05-06 15:55:54 +00:00
Rakshith R
badcac38d3 e2e: testcase for pvc-pvc clone with different SC & encryption
Signed-off-by: Rakshith R <rar@redhat.com>
2022-05-06 10:32:21 +00:00
Niels de Vos
5e66372e31 e2e: allow RWOP tests to fail
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-04-21 16:22:49 +00:00
Niels de Vos
b82af7559b e2e: add -clusterid=... for selecting a Ceph cluster
The Ceph cluster-id is usually detected with `ceph fsid`. This is not
always correct, as the the Ceph cluster can also be configured by name.
If the -clusterid=... is passed, it will be used instead of trying to
detect it with `ceph fsid`.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-04-21 16:22:49 +00:00
Niels de Vos
e30364cb4d e2e: introduce getClusterID() helper
There are many locations where the cluster-id (`ceph fsid`) is obtained
from the Rook Toolbox. Instead of duplicating the code everywhere, use a
new helper function getClusterID().

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-04-21 16:22:49 +00:00
Niels de Vos
b20e3fa784 e2e: allow passing CephFS filesystem name on CLI
A new -filesystem=... option has been added so that the e2e tests can
run against environments that do not have a "myfs" CephFS filesystem.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-04-21 16:22:49 +00:00
Niels de Vos
fcb4701a06 e2e: cleanup StorageClass create retry
StorageClasses are cluster resources, not namespaced; there is no need
to log the namespace of a StorageClass.

When creating a StorageClass, NotFound is not an error that will be
returned, not need to check for it.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-04-21 10:48:04 +00:00
Niels de Vos
b235c171ba e2e: retry creation of the RBD StorageClass
On occasion the creation of the StorageClass can fail due to an
etcdserver timeout. If that happens, the creation can be attempted after
a delay.

This has already been done for CephFS StorageClasses, but was missed for
RBD.

See-also: ceph/ceph-csi@8a0377ef02
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-04-21 10:48:04 +00:00
Niels de Vos
e3d061075c e2e: use ResourceDeployer for RBD
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-04-21 00:22:53 +00:00
Niels de Vos
60442fa916 e2e: use ResourceDeployer for CephFS
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-04-21 00:22:53 +00:00
Niels de Vos
2b4bb63eb8 e2e: introduce ResourceDeployer interface
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-04-21 00:22:53 +00:00
Niels de Vos
701b5d7ecb e2e: Add early return in Context() for disabled tests
Some parts of the Context() seem to get executed, even when BeforeEach()
did a Skip() for the test. By adding a return inside the Context(), the
tests should not get executed at all.

This was noticed in a failed test, where upgrade was running, eventhough
the job was executed as a nornal non-upgrade one.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-04-20 17:34:00 +00:00
Humble Chirammal
0dfc15121c deploy: change the image registry for sidecars
This commit change the image registry URL for sidecars in the
deployment from `k8s.gcr.io` to `registry.k8s.io` as
the migration is happening from former to the latter. This commit
also correct the e2e readme for the change.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-04-20 10:05:13 +00:00
Madhu Rajanna
b4acbd08a5 rbd: move radosNamespace to RBD section
As radosNamespace is more specific to
RBD not the general ceph configuration. Now
we introduced a new RBD section for RBD specific
options, Moving the radosNamespace to RBD section
and keeping the radosNamespace still under the
global ceph level configration for backward
compatibility.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-04-19 12:28:46 +00:00
Humble Chirammal
fcd0f4713a cleanup: correct typos in test description and source code
this commit correct typos in various places.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-04-18 10:29:08 +00:00
Madhu Rajanna
2205145654 e2e: remove claimRef after deleting the PVC
Instead of patching the PV to update
the persistentVolumeReclaimPolicy and
the claimRef before deleting the PVC.
Patch PV persistentVolumeReclaimPolicy to Retain
to retain the PV after deleting the PVC.
Remove the claimRef on the PV after deleting
the PVC so that claim can be attached to a new PVC.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-04-13 17:35:42 +00:00
Humble Chirammal
b64c7583a9 e2e: remove the release check in clone test validation
we no longer require the kubernetes validation for clone tests in
the e2e tests. This commit remove it for CephFS.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-04-13 04:55:50 +00:00
Humble Chirammal
5e23010e2b e2e: remove the release check in clone test validation
we no longer require the kubernetes validation for clone tests in
the e2e tests. This commit remove it for RBD.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-04-13 04:55:50 +00:00
Niels de Vos
5bc8584b02 e2e: add -is-openshift option to disable certain checks
On OpenShift it is not possible for the Rook toolbox to get the metrics
from Kubelet (without additional configuration). By passing
-is-openshift, the metrics are not checked, and the e2e suite does not
fail on that particular piece.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-04-12 22:43:15 +00:00
Niels de Vos
92866f46fd e2e: allow passing CephFS filesystem name on CLI
A new -filesystem=... option has been added so that the e2e tests can
run against environments that do not have a "myfs" CephFS filesystem.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-04-12 22:43:15 +00:00
Niels de Vos
ab5ca13586 e2e: return useful error message when ConfigMap creation fails
In case the toolbox pod is not available, the error message lists that
no Pods are found, but there is no hint about the toolbox. By mentioning
the toolbox in the error message, it suggests a good place to start
troubleshooting the environment.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-04-12 22:43:15 +00:00
Humble Chirammal
6043a30877 e2e: change the default release version of upgrade tests
This commit correct the release version of upgrade tests from
unsupported 3.3.1 to supported version.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-04-12 08:54:55 +00:00