Commit Graph

185 Commits

Author SHA1 Message Date
Niels de Vos
fca90695fd e2e: disable rbd-nbd tests by default
Because the rbd-nbd tests fail with minikube and the Podman driver,
disable the tests for the time being.

Updates: #3431
Signed-off-by: Niels de Vos <ndevos@redhat.com>
(cherry picked from commit 386d3ddd6e)
2022-10-13 12:43:52 +00:00
Madhu Rajanna
efabe70a46 e2e: remove kubernetes 1.22 check
We run CI jobs on kubernetes 1.22 by default
and we dont need to have a check to make sure
we have atleast Kubernetes 1.22 for few tests.
As we have CI runs on 1.22 by default, Removing
unwanted check.

updates: #3086
depends-on #3255

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-07-25 07:35:41 +00:00
Niels de Vos
2df55a55a3 e2e: use exclusive-lock together with lock_on_read
When using `lock_on_read`, the RBD image needs to have the
`exclusive-lock` feature enabled too.

Fixes: #3221
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-06-29 17:40:17 +00:00
Prasanna Kumar Kalever
b56511c0c8 e2e: reduce defaultCloneCount to 3
CI is failing very frequently hitting resource leaks issue,
until we solve the root cause for resource leaks reducing the clone
count from 10 to 3.

related: #2327
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2022-06-29 11:08:09 +00:00
Madhu Rajanna
4b57cc3ec5 rbd: add support for rbd striping
RBD supports creating rbd images with
object size, stripe unit and stripe count
to support striping. This PR adds the support
for the same.

More details about striping at
https://docs.ceph.com/en/quincy/man/8/rbd/#striping

fixes: #3124

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-06-09 18:59:00 +00:00
Prasanna Kumar Kalever
8364493eb9 e2e: add tests to validate cluster name
Validate with both volumes and snapshots

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2022-06-08 16:23:59 +00:00
Humble Chirammal
2cb52d6535 e2e: remove the generic ephemeral volume support check
at present we have the check for kube version 1.21 in the tests
which no longer required as it falls under supported kubernetes
versions with the driver.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-05-26 13:43:17 +00:00
Prasanna Kumar Kalever
ba6052e896 e2e: workaround fix for omap leaks
2 omap objects are getting leaked in the e2e tests, this change is to
workaround them for now.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2022-05-12 18:47:27 +00:00
Prasanna Kumar Kalever
dcf51a10b1 e2e: validate omap objects per rbd test case
validate omap count in every testcase right after
validateRBDImageCount()

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2022-05-12 18:47:27 +00:00
Prasanna Kumar Kalever
89a7ada12f e2e: add various testcases to validate unset of metadata
Validate that we:
* Unset the PVC metadata on the rbd image created for the snapshot
* snapshot metadata on CreateVolume from snapshot

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2022-05-12 15:54:09 +00:00
Niels de Vos
9f08cb7ea0 e2e: re-order deployment components
Some of the deployment artifacts refer to others (like ServiceAccount in
a Deployment). If the dependencies are not available (yet), there will
be errors reported in the logs. By deploying the components in a more
correct order, fewer errors are reported, making the logs a little
easier to understand.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-05-11 14:58:28 +00:00
Madhu Rajanna
89d9ec0823 e2e: add getPersistentVolumeClaim helper function
added getPersistentVolumeClaim helper function
to get the PVC and also try if there is any API
error to improve the CI.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-05-06 15:55:54 +00:00
Rakshith R
badcac38d3 e2e: testcase for pvc-pvc clone with different SC & encryption
Signed-off-by: Rakshith R <rar@redhat.com>
2022-05-06 10:32:21 +00:00
Niels de Vos
5e66372e31 e2e: allow RWOP tests to fail
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-04-21 16:22:49 +00:00
Niels de Vos
e3d061075c e2e: use ResourceDeployer for RBD
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-04-21 00:22:53 +00:00
Niels de Vos
701b5d7ecb e2e: Add early return in Context() for disabled tests
Some parts of the Context() seem to get executed, even when BeforeEach()
did a Skip() for the test. By adding a return inside the Context(), the
tests should not get executed at all.

This was noticed in a failed test, where upgrade was running, eventhough
the job was executed as a nornal non-upgrade one.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-04-20 17:34:00 +00:00
Madhu Rajanna
2205145654 e2e: remove claimRef after deleting the PVC
Instead of patching the PV to update
the persistentVolumeReclaimPolicy and
the claimRef before deleting the PVC.
Patch PV persistentVolumeReclaimPolicy to Retain
to retain the PV after deleting the PVC.
Remove the claimRef on the PV after deleting
the PVC so that claim can be attached to a new PVC.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-04-13 17:35:42 +00:00
Prasanna Kumar Kalever
4100ad2406 e2e: add test case for snapshot metadata validation
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2022-04-08 15:43:14 +00:00
Prasanna Kumar Kalever
463990c999 e2e: add test cases for image metadata validation
* create a PVC and check PVC/PV metadata on RBD image
* create and delete a PVC, attach the old PV to a new PVC and check if
  PVC metadata is updated on  RBD image

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2022-04-08 15:43:14 +00:00
Robert Vasek
548a752ff0 e2e: added test for verifying that ceph-fuse recovery works for new pods
Signed-off-by: Robert Vasek <robert.vasek@cern.ch>
2022-03-10 06:05:52 +00:00
Madhu Rajanna
d5c98f81a2 rbd: make image features as optional parameter
Makes the rbd images features in the storageclass
as optional so that default image features of librbd
can be used. and also kept the option to user
to specify the image features in the storageclass.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-02-28 13:10:03 +00:00
Madhu Rajanna
fb3835691f rbd: add support for deep-flatten image feature
as deep-flatten is long supported in ceph and its
enabled by default in the librbd, providing an option
to enable it in cephcsi for the rbd images we are
creating.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-02-28 13:10:03 +00:00
Niels de Vos
693aabbe1e e2e: use pstree instead of ps for checking rbd-nbd process
The CentOS Stream 8 base container image does not have `ps` installed.
This causes CI jobs to fail, when checking for a restarted rbd-nbd
process.

Instead of using `ps`, the `pstree` command can be used. This will add
some ASCII-tree symbols in front of the command that is logged by the
e2e tests, but that is only used for manual reviewing and does not harm
the running test.

Fixes: #2850
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-02-03 13:25:49 +00:00
Madhu Rajanna
28fef9b379 cleanup: remove thick provisioning code
This commit removes the thick provisioning
code as thick provisioning is deprecated in
cephcsi 3.5.0.

fixes: #2795

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-01-28 11:17:15 +00:00
Madhu Rajanna
aba6979d29 cleanup: use os.ReadFile to read file
as ioutil.ReadFile is deprecated and
suggestion is to use os.ReadFile as
per https://pkg.go.dev/io/ioutil updating
the same.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-01-24 05:25:11 +00:00
Humble Chirammal
8737d7e0a5 e2e: add rwop validation tests for rbd filemode and block mode
This commit adds the validation tests for rbd block and filemode
PVCs.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-01-11 19:40:22 +00:00
Madhu Rajanna
e87c0ac275 e2e: add Copyright header of e2e files
added Copyright header for files inside
the e2e folder.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-12-23 03:47:00 +00:00
Madhu Rajanna
210f95de04 e2e: add e2e for bigger size clone
added e2e for below cases

Normal PVC clone to a bigger
size PVC (without encryption)

* Filesystem pvc clone to a bigger size
* Block pvc clone to a bigger size

Encrypted PVC clone to a bigger
size PVC

* Filesystem pvc clone to a bigger size
* Block pvc clone to a bigger size

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-12-23 03:47:00 +00:00
Madhu Rajanna
494c9b91cb e2e: add e2e for bigger size restore
added e2e for below cases

Normal PVC snapshot restore to a bigger
size PVC (without encryption)

* Filesystem pvc restore to a bigger size
* Block pvc restore to a bigger size

Encrypted PVC snapshot restore to a bigger
size PVC

* Filesystem pvc restore to a bigger size
* Block pvc restore to a bigger size

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-12-23 03:47:00 +00:00
Humble Chirammal
cf6fdcb796 e2e: adjust migration e2e tests and introduce helper functions
This commit adjust existing migration e2e tests to a couple of tests
to cover the scenarios. The seperate filesystem and block tests have
been shrinked to single one and also introduced a couple of helper
functions to setup and teardown migraition specific secret,configmap
and sc. The static pv function has been renamed to a general name
while the tests were adjusted.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-12-20 07:34:43 +00:00
Madhu Rajanna
4ebfeaaa75 e2e: add e2e testcase of new image features
adding e2e testcase to validate the workflow
of pvc creation and attaching to pod works for
new image features like fast-diff,obj-map,exclusive-lock
and layering.

fixes: #2695

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-12-09 12:30:42 +00:00
Yug Gupta
2bf17007d9 e2e: validate pvc-pvc clone in ec pool
Validate pvc to pvc clone in erasure
coded pools.

Signed-off-by: Yug Gupta <yuggupta27@gmail.com>
2021-12-08 18:17:22 +00:00
Yug Gupta
938051463f e2e: validate snapshot restore in ec pool
validate snapshot restore in erasure coded
pool.

Signed-off-by: Yug Gupta <yuggupta27@gmail.com>
2021-12-08 18:17:22 +00:00
Yug Gupta
34c37663c2 e2e: Verify if app can mount ec pvc
Verify if an application is able to mount
a pvc created via an erasure coded pool.

Signed-off-by: Yug Gupta <yuggupta27@gmail.com>
2021-12-08 18:17:22 +00:00
Madhu Rajanna
d943fbd265 e2e: run generic ephemeral for kubernetes 1.21+
Currently, we are skipping the generic ephemeral
testing if the kubernetes version is less than
1.21 because of this one the who test suite is
getting skipped and e2e is marked as success
in 2 minutes. This commit runs the ephemeral
tests if the kube=>1.21+. If we do this, for
the lower version we can run other tests.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-12-07 15:02:45 +00:00
Prasanna Kumar Kalever
0fd4db92bb e2e: add tests using different accessModes and volumeModes for rbd-nbd
Add tests for RWX and ROX accessModes for Block and FileSystem Mode
PVCs.

Fixes: #2262
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-11-23 13:13:41 +00:00
Prasanna Kumar Kalever
fd7c8f375e cleanup: move waitForDeploymentComplete to deployment.go
Fixed the order of function parameters

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-11-23 13:13:41 +00:00
Prasanna Kumar Kalever
5472b66ccf e2e: start adopting to the new format of map/unmapOptions
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-11-23 08:54:37 +00:00
Humble Chirammal
4785c55bb8 e2e: reformat error message with proper error formatting
To make the error return consistent across e2e tests we have decided
to remove `with error` presence from the logs and this commit
does that for rbd tests.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-11-22 11:07:15 +00:00
Humble Chirammal
df0901ddd8 rbd: add generic ephemeral volume validation
This commit adds the validation of csi RBD driver to work with
ephemeral volume support. With ephemeral volume support a user can
specify ephemeral volumes in its pod spec and tie the lifecycle
of the PVC with the POD.

An example pod spec is also included in this commit.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-11-22 06:08:23 +00:00
Humble Chirammal
2ac3f129c0 e2e: remove 1.15 based test enablement in rbd
considering we are far out of this release and only care about
kubernetes releases from v1.20, there is no need to have this
version check in place for the tests.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-11-18 05:04:44 +00:00
Humble Chirammal
1354cfbf03 e2e: remove 1.16 based test enablement in rbd
considering we are far out of this release and only care about
kubernetes releases from v1.20, there is no need to have this
version check in place for the tests.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-11-18 05:04:44 +00:00
Humble Chirammal
c03969fa65 e2e: remove 1.17 based test enablement in rbd
considering we are far out of this release and only care about
kubernetes releases from v1.20, there is no need to have this
version check in place for the tests.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-11-18 05:04:44 +00:00
Prasanna Kumar Kalever
cee6da5313 e2e: adding io-timeout for lower kernel versions
This got removed unintentionally with
https://github.com/ceph/ceph-csi/pull/2628

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-11-16 10:12:46 +00:00
Prasanna Kumar Kalever
c97b6432e3 e2e: restrict IO with lower version kernel at rbd-nbd tests
Currently, at "perform IO on rbd-nbd volume after nodeplugin restart"
test we are performing write on the rbd-nbd based mount after nodeplugin
restart. But due to a bug in NBD driver the writes are failing, please
note NBD zero cmd timeout handling is fixed with kernel >= 5.4 and hence
we should defend on writes based on kernel version to avoid unnecessary
CI failures.

For more information see
https://github.com/ceph/ceph-csi/issues/2204#issuecomment-930941047

updates: #2204
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-11-10 16:46:50 +00:00
Humble Chirammal
de57fa1804 e2e: adjust deletion, filesystem and block tests for migration volume
this commit create and make use of migration secret in the requests and
validate various csi operations

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-10-27 18:35:00 +00:00
Rakshith R
41d894f98a e2e: add test cases for EnsureImageCleanup
This tests pvc,pvcsmartclone,snapshot deletion when
underlying images are in trash.

Signed-off-by: Rakshith R <rar@redhat.com>
2021-10-20 18:25:31 +00:00
Humble Chirammal
819f4f9048 e2e: adjust migration tests to use clusterID in the volume context
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-10-11 10:06:30 +00:00
Prasanna Kumar Kalever
a01b9821ee e2e: set rbd-nbd mounter tests cephLogStrategy to preserve
This is to preserve the rbd-nbd logs post unmap, so that the CI can dump
the available logs from logdir.

Fixes: #2451
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-10-08 14:00:42 +00:00
Rakshith R
f60b097f5f e2e: add testcase for thick encrypted PVC restore
Signed-off-by: Rakshith R <rar@redhat.com>
2021-10-05 07:46:57 +00:00