Commit Graph

518 Commits

Author SHA1 Message Date
Madhu Rajanna
fb3835691f rbd: add support for deep-flatten image feature
as deep-flatten is long supported in ceph and its
enabled by default in the librbd, providing an option
to enable it in cephcsi for the rbd images we are
creating.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-02-28 13:10:03 +00:00
Robert Vasek
e9c6fab287 e2e: loadAppDeployment now sets ImagePullPolicy to PullIfNotPresent
Fixes: #2889

Signed-off-by: Robert Vasek <robert.vasek@cern.ch>
2022-02-24 15:38:24 +00:00
Madhu Rajanna
3922cfc860 e2e: consider not found error in deployment check
it might need sometime for the deployment to
get created, consider the NotFound as a valid
error and retry again.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-02-21 12:18:39 +00:00
Niels de Vos
ea89b26f65 e2e: pass --ignore-not-found=true when deleting resources
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-02-15 16:21:07 +00:00
Niels de Vos
a669258688 cleanup: remove unused global variable in e2e
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-02-15 16:21:07 +00:00
Niels de Vos
26696b4979 e2e: retry deleting Rook artifacts in case of error
On occasion deploying CephFS components fail due to errors like these:

    failed to delete provisioner rbac .../csi-provisioner-rbac.yaml

By using the deleteResource() helper, an retry is done in case of a
failure.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-02-15 16:21:07 +00:00
Niels de Vos
8a0377ef02 e2e: retry creating StorageClass in case of error
There have been errors while CephFS tests were running, like:

    failed to create storageclass: etcdserver: request timed out

When retrying to create the StorageClass, the e2e tests are expected to
continue and (hopefully) succeed.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-02-15 16:21:07 +00:00
Niels de Vos
693aabbe1e e2e: use pstree instead of ps for checking rbd-nbd process
The CentOS Stream 8 base container image does not have `ps` installed.
This causes CI jobs to fail, when checking for a restarted rbd-nbd
process.

Instead of using `ps`, the `pstree` command can be used. This will add
some ASCII-tree symbols in front of the command that is logged by the
e2e tests, but that is only used for manual reviewing and does not harm
the running test.

Fixes: #2850
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-02-03 13:25:49 +00:00
Madhu Rajanna
28fef9b379 cleanup: remove thick provisioning code
This commit removes the thick provisioning
code as thick provisioning is deprecated in
cephcsi 3.5.0.

fixes: #2795

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-01-28 11:17:15 +00:00
Madhu Rajanna
e08084fbb6 e2e: update cephfs data pool name
cephfs data pool name is changed from filesystem-data0
to filesystem-replicated in Rook 1.8. updating
the cephcsi helper functions also to use new
pool names.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-01-27 04:18:01 +00:00
Humble Chirammal
7ff048bf1e e2e: add podsecuritycontext fsgroup for normal user validation
considering the pod has run as normal user, the fsgroup has also
set to the same.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-01-25 16:25:11 +00:00
Madhu Rajanna
aba6979d29 cleanup: use os.ReadFile to read file
as ioutil.ReadFile is deprecated and
suggestion is to use os.ReadFile as
per https://pkg.go.dev/io/ioutil updating
the same.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-01-24 05:25:11 +00:00
Madhu Rajanna
42d6130cbc e2e: cephfs e2e for bigger size clone,restore pvc
adding E2E to verify the bigger size clone
and restore pvc from a smaller size pvc/snapshot.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-01-12 10:44:11 +00:00
Humble Chirammal
0bd1d44dc0 e2e: add rwop validation for cephfs volumes
This commit adds the rwop validation tests for cephfs.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-01-11 19:40:22 +00:00
Humble Chirammal
8737d7e0a5 e2e: add rwop validation tests for rbd filemode and block mode
This commit adds the validation tests for rbd block and filemode
PVCs.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-01-11 19:40:22 +00:00
Humble Chirammal
375e9c8a51 e2e: add rwop validation helper function in pod.go
This commit adds the rwop validation helper for rbd and cephfs tests.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-01-11 19:40:22 +00:00
Humble Chirammal
180f0f660f e2e: make recreateCSIRBDPods() function to a generic one
This commit make recreateCSIRBDPods function to be a general one
so that it can be consumed by more clients.

Updates https://github.com/ceph/ceph-csi/issues/2509

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-01-04 13:48:36 +00:00
Madhu Rajanna
e87c0ac275 e2e: add Copyright header of e2e files
added Copyright header for files inside
the e2e folder.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-12-23 03:47:00 +00:00
Madhu Rajanna
210f95de04 e2e: add e2e for bigger size clone
added e2e for below cases

Normal PVC clone to a bigger
size PVC (without encryption)

* Filesystem pvc clone to a bigger size
* Block pvc clone to a bigger size

Encrypted PVC clone to a bigger
size PVC

* Filesystem pvc clone to a bigger size
* Block pvc clone to a bigger size

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-12-23 03:47:00 +00:00
Madhu Rajanna
494c9b91cb e2e: add e2e for bigger size restore
added e2e for below cases

Normal PVC snapshot restore to a bigger
size PVC (without encryption)

* Filesystem pvc restore to a bigger size
* Block pvc restore to a bigger size

Encrypted PVC snapshot restore to a bigger
size PVC

* Filesystem pvc restore to a bigger size
* Block pvc restore to a bigger size

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-12-23 03:47:00 +00:00
Humble Chirammal
cf6fdcb796 e2e: adjust migration e2e tests and introduce helper functions
This commit adjust existing migration e2e tests to a couple of tests
to cover the scenarios. The seperate filesystem and block tests have
been shrinked to single one and also introduced a couple of helper
functions to setup and teardown migraition specific secret,configmap
and sc. The static pv function has been renamed to a general name
while the tests were adjusted.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-12-20 07:34:43 +00:00
Humble Chirammal
784c3ef06d e2e: remove unparam linter escapes from getStaticPV and getStaticPVC
This `unparam` linter escape is no longer needed and CI is failing
if we keep there. This commit remove the same and make CI happy.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-12-20 07:34:43 +00:00
Madhu Rajanna
4ebfeaaa75 e2e: add e2e testcase of new image features
adding e2e testcase to validate the workflow
of pvc creation and attaching to pod works for
new image features like fast-diff,obj-map,exclusive-lock
and layering.

fixes: #2695

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-12-09 12:30:42 +00:00
Yug Gupta
2bf17007d9 e2e: validate pvc-pvc clone in ec pool
Validate pvc to pvc clone in erasure
coded pools.

Signed-off-by: Yug Gupta <yuggupta27@gmail.com>
2021-12-08 18:17:22 +00:00
Yug Gupta
938051463f e2e: validate snapshot restore in ec pool
validate snapshot restore in erasure coded
pool.

Signed-off-by: Yug Gupta <yuggupta27@gmail.com>
2021-12-08 18:17:22 +00:00
Yug Gupta
34c37663c2 e2e: Verify if app can mount ec pvc
Verify if an application is able to mount
a pvc created via an erasure coded pool.

Signed-off-by: Yug Gupta <yuggupta27@gmail.com>
2021-12-08 18:17:22 +00:00
Madhu Rajanna
d943fbd265 e2e: run generic ephemeral for kubernetes 1.21+
Currently, we are skipping the generic ephemeral
testing if the kubernetes version is less than
1.21 because of this one the who test suite is
getting skipped and e2e is marked as success
in 2 minutes. This commit runs the ephemeral
tests if the kube=>1.21+. If we do this, for
the lower version we can run other tests.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-12-07 15:02:45 +00:00
Niels de Vos
ba60566cfb e2e: retry on "transport: missing content-type field" error
The e2e sometimes fail getting objects like PVCs from the Kubernetes API
server, and log the following error:

    Error getting pvc "rbd-6940" in namespace "rbd-694": rpc error: code = Unknown desc = OK: HTTP status code 200; transport: missing content-type field

By checking the error message, and initiating a retry on this failure,
CI jobs should fail less regulary.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-11-23 17:13:56 +00:00
Prasanna Kumar Kalever
0fd4db92bb e2e: add tests using different accessModes and volumeModes for rbd-nbd
Add tests for RWX and ROX accessModes for Block and FileSystem Mode
PVCs.

Fixes: #2262
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-11-23 13:13:41 +00:00
Prasanna Kumar Kalever
e5435c5bdc e2e: add deployment example for ROX used by multiple pods mounting same device
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-11-23 13:13:41 +00:00
Prasanna Kumar Kalever
fd7c8f375e cleanup: move waitForDeploymentComplete to deployment.go
Fixed the order of function parameters

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-11-23 13:13:41 +00:00
Prasanna Kumar Kalever
6e819b60e7 e2e: add helper functions to use deployment kind objects
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-11-23 13:13:41 +00:00
Prasanna Kumar Kalever
5472b66ccf e2e: start adopting to the new format of map/unmapOptions
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-11-23 08:54:37 +00:00
Humble Chirammal
929e17d21b e2e: reformat error message with consistent formatting
To make the error return consistent across e2e tests we have decided
to remove with error presence from the logs and this commit
does that for e2e/snapshot.go.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-11-22 11:07:15 +00:00
Humble Chirammal
f7f5a41774 e2e: reformat error message with consistent formatting
To make the error return consistent across e2e tests we have decided
to remove with error presence from the logs and this commit
does that for e2e/cephfs_helper.go.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-11-22 11:07:15 +00:00
Humble Chirammal
9bab088ddc e2e: reformat error message with consistent formatting
To make the error return consistent across e2e tests we have decided
to remove with error presence from the logs and this commit
does that for e2e/upgrade-rbd.go.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-11-22 11:07:15 +00:00
Humble Chirammal
fc7d3fadf5 e2e: reformat error message with consistent formatting
To make the error return consistent across e2e tests we have decided
to remove with error presence from the logs and this commit
does that for e2e/upgrade-cephfs.go.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-11-22 11:07:15 +00:00
Humble Chirammal
5ee723e634 e2e: reformat error message with consistent formatting
To make the error return consistent across e2e tests we have decided
to remove with error presence from the logs and this commit
does that for e2e/rbd_helper.go.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-11-22 11:07:15 +00:00
Humble Chirammal
4ca19ad2ff e2e: reformat error message with consistent formatting
To make the error return consistent across e2e tests we have decided
 to remove with error presence from the logs and this commit
 does that for e2e/ceph_user.go.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-11-22 11:07:15 +00:00
Humble Chirammal
14389c7b40 e2e: reformat error message with consistent formatting
To make the error return consistent across e2e tests we have decided
to remove with error presence from the logs and this commit
does that for e2e/utils.go.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-11-22 11:07:15 +00:00
Humble Chirammal
5647d4da24 e2e: reformat error message with proper error formatting
To make the error return consistent across e2e tests we have decided
to remove `with error` presence from the logs and this commit
does that for cephfs tests.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-11-22 11:07:15 +00:00
Humble Chirammal
4785c55bb8 e2e: reformat error message with proper error formatting
To make the error return consistent across e2e tests we have decided
to remove `with error` presence from the logs and this commit
does that for rbd tests.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-11-22 11:07:15 +00:00
Humble Chirammal
e6949945bb cephfs: add validation for generic ephemeral volumes
This commit adds the validation of csi cephfs driver to work with
ephemeral volume support. With ephemeral volume support a user can
specify ephemeral volumes in its pod spec and tie the lifecycle
of the PVC with the POD.

An example POD spec also included in this commit.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-11-22 06:08:23 +00:00
Humble Chirammal
df0901ddd8 rbd: add generic ephemeral volume validation
This commit adds the validation of csi RBD driver to work with
ephemeral volume support. With ephemeral volume support a user can
specify ephemeral volumes in its pod spec and tie the lifecycle
of the PVC with the POD.

An example pod spec is also included in this commit.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-11-22 06:08:23 +00:00
Humble Chirammal
aef3cc0c3c e2e: remove 1.15 based test enablement in cephfs
Considering we are far out of these release and only care about
kubernetes releases from v1.20, there is no need to have this
version check in place for the tests.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-11-18 05:04:44 +00:00
Humble Chirammal
0c5be6d12d e2e: remove 1.16 based test enablement in cephfs
Considering we are far out of these release and only care about
kubernetes releases from v1.20, there is no need to have this
version check in place for the tests.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-11-18 05:04:44 +00:00
Humble Chirammal
7090a18141 e2e: remove 1.17 based test enablement in cephfs
Considering we are far out of these release and only care about
kubernetes releases from v1.20, there is no need to have this
version check in place for the tests.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-11-18 05:04:44 +00:00
Humble Chirammal
2ac3f129c0 e2e: remove 1.15 based test enablement in rbd
considering we are far out of this release and only care about
kubernetes releases from v1.20, there is no need to have this
version check in place for the tests.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-11-18 05:04:44 +00:00
Humble Chirammal
1354cfbf03 e2e: remove 1.16 based test enablement in rbd
considering we are far out of this release and only care about
kubernetes releases from v1.20, there is no need to have this
version check in place for the tests.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-11-18 05:04:44 +00:00
Humble Chirammal
c03969fa65 e2e: remove 1.17 based test enablement in rbd
considering we are far out of this release and only care about
kubernetes releases from v1.20, there is no need to have this
version check in place for the tests.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-11-18 05:04:44 +00:00
Niels de Vos
1fa8939e84 e2e: retry when a "transport is closing" error is hit
There have been occasional CI job failures due to "transport is closing"
errors. Adding this error to the isRetryableAPIError() function should
make sure to retry the request until the connection is restored.

Fixes: #2613
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-11-17 14:07:07 +00:00
Prasanna Kumar Kalever
0bf9db822b e2e: validate encrypted image mount inside the nodeplugin
currently the mountType validation of the encrypted volume is done in
the application, we should rather validate this inside the nodeplugin
pod.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-11-16 10:12:46 +00:00
Prasanna Kumar Kalever
cee6da5313 e2e: adding io-timeout for lower kernel versions
This got removed unintentionally with
https://github.com/ceph/ceph-csi/pull/2628

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-11-16 10:12:46 +00:00
Prasanna Kumar Kalever
c97b6432e3 e2e: restrict IO with lower version kernel at rbd-nbd tests
Currently, at "perform IO on rbd-nbd volume after nodeplugin restart"
test we are performing write on the rbd-nbd based mount after nodeplugin
restart. But due to a bug in NBD driver the writes are failing, please
note NBD zero cmd timeout handling is fixed with kernel >= 5.4 and hence
we should defend on writes based on kernel version to avoid unnecessary
CI failures.

For more information see
https://github.com/ceph/ceph-csi/issues/2204#issuecomment-930941047

updates: #2204
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-11-10 16:46:50 +00:00
Humble Chirammal
de57fa1804 e2e: adjust deletion, filesystem and block tests for migration volume
this commit create and make use of migration secret in the requests and
validate various csi operations

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-10-27 18:35:00 +00:00
Rakshith R
41d894f98a e2e: add test cases for EnsureImageCleanup
This tests pvc,pvcsmartclone,snapshot deletion when
underlying images are in trash.

Signed-off-by: Rakshith R <rar@redhat.com>
2021-10-20 18:25:31 +00:00
Humble Chirammal
819f4f9048 e2e: adjust migration tests to use clusterID in the volume context
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-10-11 10:06:30 +00:00
Prasanna Kumar Kalever
a01b9821ee e2e: set rbd-nbd mounter tests cephLogStrategy to preserve
This is to preserve the rbd-nbd logs post unmap, so that the CI can dump
the available logs from logdir.

Fixes: #2451
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-10-08 14:00:42 +00:00
Madhu Rajanna
8ebc0659ab rbd: perform resize of file system for static volume
For static volume, the user will manually mounts
already existing image as a volume to the application
pods. As its a rbd Image, if the PVC is of type
fileSystem the image will be mapped, formatted
and mounted on the node,
If the user resizes the image on the ceph cluster.
User cannot not automatically resize the filesystem
created on the rbd image. Even if deletes and
recreates the kubernetes objects, the new size
will not be visible on the node.

With this changes During the NodeStageVolumeRequest
the nodeplugin will check the size of the mapped rbd
image on the node using the devicePath. and also
the rbd image size on the ceph cluster.

If the size is not matching it will do the file
system resize on the node as part of the
NodeStageVolumeRequest RPC call.

The user need to do below operation to see new size
* Resize the rbd image in ceph cluster
* Scale down all the application pods using the static
PVC.
* Make sure no application pods which are using the
static PVC is running on a node.
* Scale up all the application pods.

Validate the new size in application pod mounted
volume.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-10-06 13:15:00 +00:00
Rakshith R
f60b097f5f e2e: add testcase for thick encrypted PVC restore
Signed-off-by: Rakshith R <rar@redhat.com>
2021-10-05 07:46:57 +00:00
Rakshith R
b471cac6bd e2e: add nolint:param to retryKubectlArgs
Currently only kubectlCreate arg is used with retryKubectlArgs(),
But it maybe used later on.

Signed-off-by: Rakshith R <rar@redhat.com>
2021-10-05 07:46:57 +00:00
Rakshith R
dac4e76ae1 e2e: add testcase for PVC restore from vaultKMS to vaultTenantSAKMS
Signed-off-by: Rakshith R <rar@redhat.com>
2021-10-05 07:46:57 +00:00
Rakshith R
f63ed2ca5a e2e: modify validatePVCSnapshot() to use restoreSCName & restoreKMS
Signed-off-by: Rakshith R <rar@redhat.com>
2021-10-05 07:46:57 +00:00
Humble Chirammal
b778fe51a4 e2e: add test for migration volID detection and delete of image
This commit add test for migration delete volID detection scenario
by passing a custom volID and with the entries in configmap changed
to simulate the situation. The staticPV function also changed its
accept the annotation map which make it more general usage.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-10-04 16:06:31 +00:00
Humble Chirammal
1171111a94 e2e: deletePodWithLabel fails on unparam linter
this commit address the unparam linter error on deletePodWithLabel
function.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-10-04 16:06:31 +00:00
Humble Chirammal
a4a2dc93c1 e2e: change createCustomConfigmap to be more general
createCustomConfigmap helps to create a custom cluster entry in
the configmap, however this was coupled with subvolumegroup filling
in the cluster configuration. This commit helps to make it more
general and the subvolumegroup filling is controlled now with a flag

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-10-04 16:06:31 +00:00
Yati Padia
1cf14cd83c cleanup: rework on naming conventions
This commits replaces cephfs -> cephFS
to maintain consistency throughout the
codebase

Updates: #1465

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-09-24 06:17:17 +00:00
Humble Chirammal
4804f47b18 e2e: Add e2e for rbd migration static pvc
This commit adds e2e for rbd migration static PVCs

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-09-20 09:54:54 +00:00
Madhu Rajanna
5a53a9ad76 e2e: update e2e deployment for ceph conf
updated e2e deployment code to create and
delete the ceph conf configmap.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-09-03 14:14:43 +00:00
Rakshith R
f9d4972444 e2e: fix log msg in retryKubectlInput()
e2elog.Logf("waiting for kubectl (%s -f $q args %s) to finish", action, args)
changed to
e2elog.Logf("waiting for kubectl (%s -f args %s) to finish", action, args)

Signed-off-by: Rakshith R <rar@redhat.com>
2021-08-27 07:14:10 +00:00
Rakshith R
dbf2eb3905 e2e: ignore lines with 'Warning' in isAlreadyExistsCliError()
Signed-off-by: Rakshith R <rar@redhat.com>
2021-08-27 07:14:10 +00:00
Prasanna Kumar Kalever
55d3226d6b e2e: use io-timeout conditionally based on kernel version
We need
https://www.mail-archive.com/linux-block@vger.kernel.org/msg38060.html
inorder to use `--io-timeout=0`. This patch is part of kernel 5.4

Since minikube doesn't have a v5.4 kernel yet, lets use io-timeout value
conditionally based on kernel version at our e2e.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-08-24 17:09:09 +00:00
Prasanna Kumar Kalever
1bd2d46cdb e2e: add util to get kernel version from specified container
Currently, we get the kernel version where the e2e (client) executable runs,
not the kernel version that is used by the csi-rbdplugin pod.

Add a function that run `uname -r` command from the specified container and
returns the kernel version.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
Suggested-by: Niels de Vos <ndevos@redhat.com>
2021-08-24 17:09:09 +00:00
Prasanna Kumar Kalever
18f4a51a15 e2e: improve the debug logs for rbd-nbd
Ceph’s logging levels operate on a scale of 1 to 20, where 1 is terse
and 20 is verbose.

Format:
debug-{subsystem} = {log-level}

Setting `rbd` loglevel to 20 at our e2e tests.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-08-19 20:16:24 +00:00
Prasanna Kumar Kalever
2723353f8d e2e: add testcase for encrypted volume with rbd-nbd mounter
Fixes: #2235

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-08-11 04:10:34 +00:00
Prasanna Kumar Kalever
9669394b23 e2e: add testcase for volume expansion with rbd-nbd mounter
The rbd-nbd resize volume support with its netlink interface needs linux
kernel version >= v5.3.0

Hence define a defence check for the supported kernel version

Fixes: #2234

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-08-10 13:21:16 +00:00
Rakshith R
1d49b6a288 e2e: create reusable variable vaultUserSecretPath = "user-secret.yaml"
Signed-off-by: Rakshith R <rar@redhat.com>
2021-08-10 09:17:59 +00:00
Rakshith R
2f995eada2 e2e: add modification to test encrypted PVC with rbd controller
Signed-off-by: Rakshith R <rar@redhat.com>
2021-08-10 09:17:59 +00:00
Rakshith R
8ca7a35820 e2e: use retryKubectlFile() for creating & deleting secrets
Signed-off-by: Rakshith R <rar@redhat.com>
2021-08-10 09:17:59 +00:00
Rakshith R
0744ad502b e2e: add prefixname to rbd controller test
Signed-off-by: Rakshith R <rar@redhat.com>
2021-08-10 09:17:59 +00:00
Rakshith R
7f6b73e71f e2e: log imageList in validateRBDImageCount for better debugging
Signed-off-by: Rakshith R <rar@redhat.com>
2021-08-09 07:28:43 +00:00
Rakshith R
9d57717222 e2e: add test cases for pvc-pvcClone chain with depth 2
Signed-off-by: Rakshith R <rar@redhat.com>
2021-08-09 07:28:43 +00:00
Rakshith R
9321b4bce4 e2e: add test cases for snapshot-restore chain with depth 2
Signed-off-by: Rakshith R <rar@redhat.com>
2021-08-09 07:28:43 +00:00
Niels de Vos
bb60173a98 e2e: add verifyKeyDestroyed() for validating vaultDestroyKeys
The kmsConfig type in the e2e suite has been enhanced with two functions
that make it possible to validate the destruction of deleted keys.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-08-06 12:19:18 +00:00
Madhu Rajanna
2c66dfc3e4 e2e: retry running kubectl on known errors
By using retryKubectl helper function,
a retry will be done, and the known error
messages will be skipped.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-08-06 08:03:18 +00:00
Madhu Rajanna
2071c535fa e2e: pass variadic argument to kubectl helper function
this provides caller ability to pass the arguments
like ignore-not-found=true etc when executing
the kubectl commands.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-08-06 08:03:18 +00:00
Madhu Rajanna
9f0af30735 e2e: add retryKubectlArgs helper for kubectl retry
added helper function retryKubectlArgs to perform
action if its a known error.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-08-06 08:03:18 +00:00
Madhu Rajanna
dd9fabf747 e2e: add isAlreadyExistsCLIError to check known error
added isAlreadyExistsCLIError to check for known error.
if error is already exists we are considering it
as a success.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-08-06 08:03:18 +00:00
Niels de Vos
72d56cb8db e2e: use original namespace for retrying resize check
expandPVCSize() uses the namespace of the PVC that was checked. In case
the .Get() call fails, the PVC will not have its namespace set, and
subsequent tries will fail with errors like:

    Error getting pvc in namespace: '': etcdserver: request timed out
    waiting for PVC  (9 seconds elapsed)
    Error getting pvc in namespace: '': an empty namespace may not be set when a resource name is provided

By using the original namespace of the PVC stored in a separate variable
as is done with the name of the PVC, this problem should not occur
anymore.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-08-04 08:08:24 +00:00
Niels de Vos
a7ff868dae e2e: retry getting the Services for Ceph MON on failures
In case listing the Kubernetes Services fails, the following error is
returned immediately:

    failed to create configmap with error failed to list services: etcdserver: request timed out

Wrapping the listing of the Services in a PollImmediate() routine, adds
a retry in case of common temporary issues.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-08-03 18:44:03 +00:00
Niels de Vos
e0ac70f8fb e2e: use official CentOS container location
registry.centos.org is not officially maintained by the CentOS
infrastructure team. The container images on quay.io are the official
once and we should use those instead.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-08-03 12:19:46 +00:00
Prasanna Kumar Kalever
c9cd8d7a37 e2e: sync data from rbd-nbd mount
Until we have a real fix, just to avoid occasionally file system entering
into read-only on nodeplugin restart, lets sync data from the application
pod.

Updates: #2204

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-07-30 15:39:48 +00:00
Niels de Vos
d3beaeb014 e2e: retry deploying CephFS components on failure
There are reports where CephFS deploying failed with etcdserver
timeouts:

    INFO: Running '/usr/bin/kubectl --server=https://192.168.39.187:8443 --kubeconfig=/root/.kube/config --namespace=cephcsi-e2e-ea434921 create --namespace=cephcsi-e2e-ea434921 -f -'
    INFO: rc: 1
    FAIL: failed to create CephFS provisioner rbac with error error running /usr/bin/kubectl --server=https://192.168.39.187:8443 --kubeconfig=/root/.kube/config --namespace=cephcsi-e2e-ea434921 create --namespace=cephcsi-e2e-ea434921 -f -:
    Command stdout:
    role.rbac.authorization.k8s.io/cephfs-external-provisioner-cfg created
    rolebinding.rbac.authorization.k8s.io/cephfs-csi-provisioner-role-cfg created

    stderr:
    Error from server: error when creating "STDIN": etcdserver: request timed out
    Error from server: error when creating "STDIN": etcdserver: request timed out
    Error from server: error when creating "STDIN": etcdserver: request timed out

    error:
    exit status 1

By using retryKubectlInput() helper function, a retry will be done, and
the failure should not be fatal any longer.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-07-29 12:35:52 +00:00
Madhu Rajanna
080b251850 e2e: validate images in trash for rados namespace
added validation check to verify stale images in trash
for the rados namespace testing.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-07-28 03:48:33 +00:00
Niels de Vos
ec6703ed58 rbd: rename encryption metadata keys to enable mirroring
RBD image metadata keys that start with '.rbd' are expected to be
internal to RBD itself and are not mirrored to remote sites. Renaming
the keys (dropping the '.' prefix) and using the new MigrateMetadata()
function now makes the keys available on remote sites too.

Closes: #2219
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-07-26 11:49:56 +00:00
Niels de Vos
e75d308b9c e2e: isRetryableAPIError() should match any etcdserver timeout
framework.RunKubectl() returns an error that does not end with
"etcdserver: request timed out", but contains the text somewhere in the
middle:

    error running /usr/bin/kubectl --server=https://192.168.39.57:8443 --kubeconfig=/root/.kube/config --namespace=cephcsi-e2e-a44ec4b4 create -f -:
    Command stdout:

    stderr:
    Error from server: error when creating "STDIN": etcdserver: request timed out

    error:
    exit status 1

isRetryableAPIError() should  return `true` for this case as well, so
instead of using HasSuffix(), we'll use Contains().

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-07-23 12:20:16 +00:00
Prasanna Kumar Kalever
75dda7ac0d e2e: add test for expansion of encrypted volumes
Also adds a test case to validate the default encryption type

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-07-23 10:00:23 +00:00
Yati Padia
1ae2afe208 cleanup: modifies the error caused due to merged PRs
This commit modifies the error of godot, cyclop,
paralleltest linter caused due to merged PRs.

Updates: #1586

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-07-22 18:15:48 +00:00
Yati Padia
172b66f73f cleanup: resolves cyclop linter issue
this commit adds `// nolint:cyclop` for the
fucntions whose complexity is above 20

Updates: #1586

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-07-22 18:15:48 +00:00
Rakshith R
43f753760b cleanup: resolve nlreturn linter issues
nlreturn linter requires a new line before return
and branch statements except when the return is alone
inside a statement group (such as an if statement) to
increase code clarity. This commit addresses such issues.

Updates: #1586

Signed-off-by: Rakshith R <rar@redhat.com>
2021-07-22 06:05:01 +00:00
Niels de Vos
5c016b4b94 e2e: retry on "connect: connection refused" errors
Sometimes there are failures in the e2e suite when connecting to the
etcdserver fails. The following error was caught:

    INFO: Error getting pvc "rbd-pvc" in namespace "rbd-1318": Get "https://192.168.39.222:8443/api/v1/namespaces/rbd-1318/persistentvolumeclaims/rbd-pvc": dial tcp 192.168.39.222:8443: connect: connection refused
    FAIL: failed to create PVC with error failed to get pvc: Get "https://192.168.39.222:8443/api/v1/namespaces/rbd-1318/persistentvolumeclaims/rbd-pvc": dial tcp 192.168.39.222:8443: connect: connection refused

If etcdserver was only briefly unavailable, one or more retries might be
sufficient to have the test pass.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-07-21 13:08:41 +00:00
Yati Padia
7f5df7c940 cleanup: resolves gofumpt issues in e2e
This commit resolves gofumpt issues in
e2e folder.

Updates: #1586

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-07-20 15:37:58 +00:00
Niels de Vos
841a53bc3d e2e: retry kubectl commands in case deploying Vault fails
Sometimes it happens that the deployment of Hashicorp Vault fails.
Deployment is one of the 1st steps that are done when starting the e2e
suite, and the Kubernetes cluster may still be a little overloaded while
it is settling down. It should be possible to retry and succeed after a
while.

Fixes: #2288
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-07-19 16:12:18 +00:00
Niels de Vos
d5ea89e603 e2e: add retryKubectlInput() for retrying kubectl calls
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-07-19 16:12:18 +00:00
Yati Padia
3469dfc753 cleanup: resolve errorlint issues
This commit resolves errorlint issues
which checks for the code that will cause
problems with the error wrapping scheme.

Updates: #1586

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-07-19 13:31:29 +00:00
Yati Padia
bfda5fa57f cleanup: resolve revive linter issue
revive linter checks for var-declaration
format.
For example:
"e2e/rbd_helper.go:441:36: var-declaration:
should drop = nil from declaration of
var noPVCValidation; it is the zero value (revive)
var noPVCValidation validateFunc = nil"

Updates: #1586

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-07-19 08:39:32 +00:00
Prasanna Kumar Kalever
10fc639d68 ci: fix nolintlint warnings
warnings from golangci-lint:

e2e/pod.go:207:122: directive `//nolint:unparam,lll // cn can be used
with different inputs later` is unused for linter unparam (nolintlint)
func execCommandInContainer(f *framework.Framework, c, ns, cn string,
opt *metav1.ListOptions) (string, string, error) { //nolint:unparam,lll
// cn can be used with different inputs later

e2e/pod.go:307:70: directive `//nolint:unparam // skipNotFound can be
used with different inputs later` is unused for linter unparam (nolintlint)
func deletePodWithLabel(label, ns string, skipNotFound bool) error {
//nolint:unparam // skipNotFound can be used with different inputs later

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-07-16 16:30:58 +00:00
Prasanna Kumar Kalever
fd3bf1750b e2e: fix the testcases for rbd-nbd
Now that the healer functionaity for mounter processes is available,
lets start, using it.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-07-16 16:30:58 +00:00
Niels de Vos
4d4a2a7814 e2e: prevent re-using empty pvc object
When an error occurs, the pvc object is overwritten in the
PollImmediate() loop. Re-using the pvc.Namespace results in error
messages like

    Error getting pvc in namespace: '': an empty namespace may not be set when a resource name is provided

and prevents the retry by PollImmediate() to never succeed. Storing the
namespace in a local variable prevents this from happening.

Reported-by: Rakshith R <rar@redhat.com>
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-07-14 10:18:51 +00:00
Niels de Vos
f7ae33c67c e2e: only call error check functions when err != nil
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-07-14 10:18:51 +00:00
Niels de Vos
075a4087d7 e2e: mark "etcdserver: request timed out" errors as retryable
There are regular CI failures where etcdserver times out. These errors
seem not to get caught by any of the existing error comparing. Matching
the error by string should prevent temporary etcdserver issues now too.

Updates: #2218
Closes: #1969
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-07-14 10:18:51 +00:00
Niels de Vos
e3c7dea7d6 e2e: add test for Vault with ServiceAccount per Tenant
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-07-13 17:16:35 +00:00
Yati Padia
4a649fe17f cleanup: resolve godot linter
This commit resolves godot linter issue
which says "Comment should end in a period (godot)".

Updates: #1586

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-07-13 06:50:03 +00:00
Niels de Vos
855f9080d1 e2e: refactor KMS configuration and usage
This adds a new `kmsConfig` interface that can be used to validate
different KMS services and setting. It makes checking for the available
support easier, and fetching the passphrase simpler.

The basicKMS mirrors the current validation of the KMS implementations
that use secrets and metadata. vaultKMS can be used to validate the
passphrase stored in a Vault service.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-07-12 14:57:39 +00:00
Yati Padia
84c1fe52c7 cleanup: resolve exhaustive linter
This commit resolves exhaustive linter
error.

Updates: #2240

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-07-12 04:47:08 +00:00
Rakshith R
b27d6319ca e2e: add e2e for user secret based metadata encryption
This commit adds e2e for user secret based metadata encryption,
adds user-secret.yaml and makes required changes in kms-connection-details,
kms-config yamls.

Signed-off-by: Rakshith R <rar@redhat.com>
2021-07-08 17:06:02 +00:00
Yati Padia
ffab37f44f cleanup: resolves gocritic linter issues
This commit resolves gocritic linter errors.

Updates: #2250

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-07-08 05:19:26 +00:00
Madhu Rajanna
f471a206c5 e2e: update e2e to validate thick metadata
updated e2e tests to validate the thick
metadata set on the rbd image for thick PVC.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-07-07 08:31:10 +00:00
Yug
e14d649547 e2e: add cephfs test for sc, secret via helm
add cephfs test for validating storageclass and
secret deployment via helm.

Signed-off-by: Yug <yuggupta27@gmail.com>
2021-07-06 10:55:41 +00:00
Yug
7d28896e36 e2e: add rbd test for testing sc, secret via helm
add rbd test for validating storageclass and
secret deployment via helm.

Signed-off-by: Yug <yuggupta27@gmail.com>
2021-07-06 10:55:41 +00:00
Rakshith R
fb365005da e2e: update e2epv.WaitOnPVandPVC() to match new parameters
Added &framework.TimeoutContext{ClaimBound: timeout, PVBound: timeout}
to parameters of function call in updated package.

Signed-off-by: Rakshith R <rar@redhat.com>
2021-07-01 03:35:23 +00:00
Madhu Rajanna
3e9172ab31 e2e: check isRetryableAPIError during polling
check isRetryableAPIError error when the
function is polling for the resouce.

updates: #1969

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-06-30 10:00:44 +00:00
Yati Padia
3c773b24e5 e2e: validate encryption keys in KMS
this commit is to validate if the encrypted
keys are created and deleted properly while
pvc-pvc clone images

Updates: #2022

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-06-30 08:21:01 +00:00
Humble Chirammal
36f4c0cabb e2e: remove passing waitgroups for go routines
We are unwantedly passing around waitgroups which is not required.
This address the same inside e2e/cephfs.go and remove waitgroup arg

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-06-29 15:07:56 +00:00
Humble Chirammal
4657ac7f00 e2e: remove passing waitgroups for go routines
We are unwantedly passing around waitgroups which is not required.
This address the same inside e2e/utils.go and remove waitgroup arg

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-06-29 15:07:56 +00:00
Humble Chirammal
053f7ee213 e2e: remove passing waitgroups reference for go routines
We are unwantedly passing around waitgroups which is not required.
This address the same inside e2e/rbd_helper.go and remove the waitgroup
arg

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-06-29 15:07:56 +00:00
Yati Padia
7f7a41d23a cleanup: use %w to format errors
Use %w instead of %v to format errors.

Updates: #1586

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-06-29 13:05:31 +00:00
Yati Padia
c851b69160 cleanup: error is not nil but returns nil
In few places the error is not nil still it
returns nil.

Updates: #1586

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-06-28 17:54:38 +00:00
Humble Chirammal
c2cda4909b e2e: reformat long lines in this package to 120 chars
We have many declarations and invocations..etc with long lines which are
very difficult to follow while doing code reading. This address the issues
in 'e2e/upgrade_*.go' files to restrict the line length to 120 chars.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-06-28 14:43:49 +00:00
Humble Chirammal
41ecaadc82 e2e: reformat long lines in this package to 120 chars
We have many declarations and invocations..etc with long lines which are
very difficult to follow while doing code reading. This address the issues
in 'e2e/staticpvc.go' and 'e2e/snapshot.go' files to restrict the line length
to 120 chars.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-06-28 14:43:49 +00:00
Humble Chirammal
d6a4003981 e2e: reformat long lines in this package to 120 chars
We have many declarations and invocations..etc with long lines which are
very difficult to follow while doing code reading. This address the issues
in 'e2e/utils.go' files to restrict the line length to 120 chars.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-06-28 14:43:49 +00:00
Humble Chirammal
7aa53350ff e2e: reformat long lines in this package to 120 chars
We have many declarations and invocations..etc with long lines which are
very difficult to follow while doing code reading. This address the issues
in 'e2e/resize.go' files to restrict the line length to 120 chars.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-06-28 14:43:49 +00:00
Humble Chirammal
1849887b40 e2e: reformat long lines in this package to 120 chars
We have many declarations and invocations..etc with long lines which are
very difficult to follow while doing code reading. This address the issues
in 'e2e/pvc.go' file to restrict the line length to 120 chars.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-06-28 14:43:49 +00:00
Humble Chirammal
676b9f84c3 e2e: reformat long lines in this package to 120 chars
We have many declarations and invocations..etc with long lines which are
very difficult to follow while doing code reading. This address the issues
in 'e2e/pod.go' files to restrict the line length to 120 chars.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-06-28 14:43:49 +00:00
Humble Chirammal
abc993ce25 e2e: reformat long lines in this package to 120 chars
We have many declarations and invocations..etc with long lines which are
very difficult to follow while doing code reading. This address the issues
in 'e2e/deploy_vault.go' file to restrict the line length to 120 chars.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-06-28 14:43:49 +00:00
Humble Chirammal
971f8e320b e2e: reformat long lines in this package to 120 chars
We have many declarations and invocations..etc with long lines which are
very difficult to follow while doing code reading. This address the issues
in 'e2e/rbd_helper.go' files to restrict the line length to 120 chars.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-06-28 14:43:49 +00:00
Humble Chirammal
f07c5ab857 e2e: reformat long lines in this package to 120 chars
We have many declarations and invocations..etc with long lines which are
very difficult to follow while doing code reading. This address the issues
in 'e2e/rbd*.go' files to restrict the line length to 120 chars.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-06-28 14:43:49 +00:00
Humble Chirammal
49d11bf5ea e2e: reformat long lines in this package to 120 chars
We have many declarations and invocations..etc with long lines which are
very difficult to follow while doing code reading. This address the issues
in 'e2e/cephfs*.go' files to restrict the line length to 120 chars.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-06-28 14:43:49 +00:00
Rakshith R
82a204bfd3 e2e: add e2e for static PVC without imageFeature parameter
This commit adds e2e to make sure static PVC without imageFeatures
parameter fail with proper error event.

Signed-off-by: Rakshith R <rar@redhat.com>
2021-06-28 05:46:42 +00:00
Yati Padia
13667c013c cleanup: addresses paralleltest linter
The Go linter paralleltest checks that the t.Parallel
gets called for the test method and for the range of
test cases within the test.

Updates: #2025

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-06-25 11:55:12 +00:00
Humble Chirammal
0749315d7e e2e: make use of snapshot v1 api in utils.go
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-06-24 17:35:07 +00:00
Humble Chirammal
7823b360b8 e2e: make use of snapshot v1 api in cephfs_helper.go
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-06-24 17:35:07 +00:00
Humble Chirammal
f3d400730e e2e: make use of snapshot v1 api in cephfs.go
The import name of snapshot api package has also been changed
from `vs` to `snapapi` for consistency.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-06-24 17:35:07 +00:00
Humble Chirammal
5ba7790b26 e2e: make use of snapshot v1 api in rbd_helper.go
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-06-24 17:35:07 +00:00
Humble Chirammal
b7cd946b1e e2e: use snapshot v1 clientset and apis in snapshot.go
snapshot.go currently make use of snapshot v1beta1 clientset and api,
with this commit it has been rolled into v1 clientset and api.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-06-24 17:35:07 +00:00
Yug
a4548c3983 e2e: provide an option if tests run on helm deployment
add an e2eArg `helmTest` to specify if tests are running
on ceph-csi deployment via helm.
For testing in CI, Storageclass and secret deployment
is enabled on helm installation.

Signed-off-by: Yug <yuggupta27@gmail.com>
2021-06-24 11:51:56 +00:00
Humble Chirammal
8639015a9f cleanup: correct initResources() function name typo
This function was wrongly declared with name initResouces() in e2e
utils package and this patch address the typo in the name

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-06-23 12:15:40 +00:00
Humble Chirammal
9b6b83f73b e2e: remove bootstrapper BeforeEach and AfterEach
The e2e bootstrap does not make use of these or its declared
unwantedly in the same, removing it with this commit.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-06-22 14:15:14 +00:00
Yati Padia
847b996501 cleanup: Modifies Wrapcheck linter
Wrapcheck is a  simple Go linter to check that errors
from external packages are wrapped during return to
help identify the error source during debugging.
This commit addresses the wrapcheck error

Updates:#2025

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-06-22 08:47:55 +00:00
Niels de Vos
0fe0962dc1 e2e: validate thick-provisioned PVC-PVC cloning
Add a case to create a new PVC with VolumeContentSource set to a
thick-provisioned PVC. This should result in a new thick-provisioned PVC
once the cloning is done.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-06-18 06:25:56 +00:00
Madhu Rajanna
c6bc84d847 e2e: validate images in trash
when all the PVC and associated images are deleted,
the images should also get deleted from the trash.
This commit adds the validation check for the same.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-06-15 15:38:24 +00:00
Humble Chirammal
4302e5ee67 cleanup: correct createORDeleteCephfsResources() function name
Along with correcing the name of the function, other typos are also
addressed

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-06-11 08:04:07 +00:00
Humble Chirammal
a9fa108959 e2e: use proper variable name for rbd mount options
The variable naming for rbd mount options has been changed
to rbdMountOptions to be consistent with other variable naming schema

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-06-11 08:04:07 +00:00
Humble Chirammal
5635e99e37 e2e: correct gosec marker for credentials rule
The marker for hardcoded credentials check was set wrongly
and this patch address the same

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-06-11 08:04:07 +00:00
Humble Chirammal
2851c7d17a cleanup: correct createORdeleteRbdResources() function name
This patch address a typo in the mentioned function name

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-06-11 08:04:07 +00:00
Yug
33bf3fae6b e2e: add logErr function to log errors
logErr function logs all the ocured errors
with a message that is passed for occurence
of each error.

Co-authored-by: Niels de Vos <ndevos@redhat.com>
Co-authored-by: Madhu Rajanna <madhupr007@gmail.com>
Signed-off-by: Yug <yuggupta27@gmail.com>
2021-06-10 21:46:53 +00:00
Yug
8d038db047 e2e: add e2e to test clone in different pool
added a e2e to test clones in different
pool.

Co-authored-by: Madhu Rajanna <madhupr007@gmail.com>
Signed-off-by: Yug <yuggupta27@gmail.com>
2021-06-10 21:46:53 +00:00
Yug
1b2e9f556f e2e: add helper function for clone validation
added a helper function to test clone creation
in a different pool.

Co-authored-by: Madhu Rajanna <madhupr007@gmail.com>
Signed-off-by: Yug <yuggupta27@gmail.com>
2021-06-10 21:46:53 +00:00
Yug
91662b54b6 e2e: move yaml filepath declaration to global
declared yaml filepath in gobal for reusing.

Co-authored-by: Madhu Rajanna <madhupr007@gmail.com>
Signed-off-by: Yug <yuggupta27@gmail.com>
2021-06-10 21:46:53 +00:00
Yug
30092ff4e1 e2e: add an option to set rbd storageclass name
added an option to set storageclass name.

Co-authored-by: Madhu Rajanna <madhupr007@gmail.com>
Signed-off-by: Yug <yuggupta27@gmail.com>
2021-06-10 21:46:53 +00:00
Yug
76e9b4b02a e2e: add function to create rbd pool
added a new function to create rbd pool
in e2e.

Co-authored-by: Madhu Rajanna <madhupr007@gmail.com>
Signed-off-by: Yug <yuggupta27@gmail.com>
2021-06-10 21:46:53 +00:00
Yug
e302797217 e2e: update listRBDImages to list images in different pool
made pool as a argument of listRBDImages to support
listing of rbd images in different pools.

Co-authored-by: Madhu Rajanna <madhupr007@gmail.com>
Signed-off-by: Yug <yuggupta27@gmail.com>
2021-06-10 21:46:53 +00:00
Yati Padia
21a400839f cleanup: No use of variable validateEncryption
In the function validatePVCSnapshot(...), we don't need
validateEncryption variable as we are passing kms value
which can help us check the value of validateEncryption.
Hence, we can avoid using that.

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-06-08 13:00:11 +00:00
Yati Padia
0f44c6acb7 cleanup: address wasted assign issues
At places variable is reassigned without
being used.

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-06-03 09:51:14 +00:00
Rakshith R
81809500be e2e: wait for upgraded pods to be running during upgrade-tests
This commit calls `waitForDaemonSets` and `waitForDeploymentComplete`
after upgrading to wait for csi driver pods to be in running state
for both rbd and cephfs upgrade tests.

Signed-off-by: Rakshith R <rar@redhat.com>
2021-05-26 13:15:52 +00:00
Prasanna Kumar Kalever
6984da5096 build: ignore unparam linter false positive
Ignoring below warnings:

e2e/pod.go:207:60: `execCommandInContainer` - `cn` always receives
`"csi-rbdplugin"` (unparam)
func execCommandInContainer(f *framework.Framework, c, ns, cn string,
opt *metav1.ListOptions) (string, string, error) {
                                                           ^
e2e/pod.go:308:43: `deletePodWithLabel` - `skipNotFound` always receives
`false` (unparam)
func deletePodWithLabel(label, ns string, skipNotFound bool) error {

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-05-26 10:41:34 +00:00
Prasanna Kumar Kalever
85e1e0370a e2e: enable an old testcase as the ndb module is available
This testcase tests journaling/exclusive-lock image-features with
rbd-nbd mounter

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-05-26 10:41:34 +00:00
Prasanna Kumar Kalever
819267112e e2e: restart rbd-nbd process after nodeplugin reboot
Bringup the rbd-nbd map/attach process on the rbd node plugin and expect the
IO to continue uninterrupted.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-05-26 10:41:34 +00:00
Prasanna Kumar Kalever
7334c3b783 e2e: add ability to run command inside specified container
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-05-26 10:41:34 +00:00
Prasanna Kumar Kalever
695ec6dffe e2e: Test IO after nodeplugin reboot
This is a negative testcase to showcase as per current design
the IO will fail because of the missing mappings

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-05-26 10:41:34 +00:00
Prasanna Kumar Kalever
8bae8f8458 e2e: add a test case for rbd-nbd mounter
To validate the basic working of rbd-nbd

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-05-26 10:41:34 +00:00
Rakshith R
fa1414d98f cleanup: address ineffectual assignement linter issue
updates: #1586

Signed-off-by: Rakshith R <rar@redhat.com>
2021-05-26 07:04:32 +00:00
Rakshith R
b891e5585d cleanup: address ifshort linter issues
This commit addresses ifshort linter issues which
checks if short syntax for if-statements is possible.

updates: #1586

Signed-off-by: Rakshith R <rar@redhat.com>
2021-05-26 07:04:32 +00:00
Niels de Vos
2b9f6c3598 e2e: fetch volume metrics from Kubelet
Test if metrics are available at all. The actual values are a little
difficult to validate.

BlockMode volumes support metrics since Kubernetes 1.22.

See-also: kubernetes/kubernetes#97972
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-05-25 06:41:04 +00:00
Humble Chirammal
151b8c665f e2e: update upgrade-version to v3.3.1
making default version of upgrade test to v3.3.1

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-05-24 16:12:20 +00:00
Madhu Rajanna
fa36a46682 e2e: pvc mounting when snap and parent pvc is deleted
Added an E2E test to test below case

* Create PVC
* Create Snapshot from PVC
* Delete PVC
* Create Clone from Snapshot
* Delete Snapshot
* Mount clone to Application
* Delete Application and PVC Clone

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-05-07 07:57:37 +00:00
Madhu Rajanna
c3bae17fce rbd: delete encryption key from KMS
when a Snapshot is encrypted during a CreateSnapshot
operation, the encryption key gets created in the KMS
when we delete the Snapshot the key from the KMS
should also gets deleted.

When we create a volume from snapshot we are copying
required information but we missed to copy the
encryption information, This commit adds the missing
information to delete the encryption key.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-04-30 08:05:47 +00:00
Niels de Vos
35d58a7d5a e2e: only test a single encrypted clone/snapshot
The default number for cloning and snapshot/restore is 10 volumes. This
adds to the time the test suite runs. There is no need to validate 10
copies of the encrypted volume, a single copy is sufficient.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-04-14 03:59:28 +00:00
Niels de Vos
bb296c4f21 e2e: add verification for encrypted Snapshot/Restore operations
This moves validatePVCSnapshot() into its own function, so that it
follows the same format as validatePVCClone() does already.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-04-14 03:59:28 +00:00
Niels de Vos
3fde636685 e2e: add validation for cloning encrypted volumes
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-04-14 03:59:28 +00:00
Rakshith R
3f3489367c cleanup: correct linter name mnd to gomnd
Signed-off-by: Rakshith R <rar@redhat.com>
2021-04-09 07:24:47 +00:00
Yug
4f955e474d cleanup: correct misspelling
Correct snapshots spelling in rbd.go

Signed-off-by: Yug <yuggupta27@gmail.com>
2021-04-01 12:00:21 +00:00
Madhu Rajanna
fba6a2d0c3 deploy: add csidriver object for cephfs and rbd
csidriver object can be created on the kubernetes
for below reason.

If a CSI driver creates a CSIDriver object,
Kubernetes users can easily discover the CSI
Drivers installed on their cluster
(simply by issuing kubectl get CSIDriver)

Ref: https://kubernetes-csi.github.io/docs/csi-driver-object.html#what-is-the-csidriver-object

attachRequired is always required to be set to
true to avoid issue on RWO PVC.

more details about it at https://github.com/rook/rook/pull/4332

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-03-31 13:41:35 +00:00
Niels de Vos
296f751d08 e2e: verify (non)existence of keys for VaultTokensKMS
Key existence and removal is only checked for the VaultKMS provider. It
should also be done for the VaultTokensKMS provider.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-03-30 08:29:33 +00:00
Rakshith R
9e957520ae e2e: add test case for app deletion with unmounted cephFS volume
Checks app deletion when cephFS volume is already unmounted.
Creates app, pvc and binds them. Unmounts the volume through
umount cmd in cephFS plugin and checks app deletion.

Signed-off-by: Rakshith R <rar@redhat.com>
2021-03-25 15:15:07 +00:00
Rakshith R
e2dc8439e4 e2e: add execCommandInDaemonsetPod()
execCommandInDaemonsetPod() executes commands inside given
container of a daemonset pod on a particular node.

Signed-off-by: Rakshith R <rar@redhat.com>
2021-03-25 15:15:07 +00:00
Rakshith R
915437b314 e2e: add listPods()
listPods returns slice of pods matching given ListOptions and
namespace.

Signed-off-by: Rakshith R <rar@redhat.com>
2021-03-25 15:15:07 +00:00
Prasanna Kumar Kalever
e6a0049c0b e2e: add getDaemonSetLabelSelector() to utils
getDaemonSetLabelSelector returns labels of daemonset given name and
namespace dynamically, needed since labels are not same for helm and
non-helm deployments.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
Signed-off-by: Rakshith R <rar@redhat.com>
2021-03-25 15:15:07 +00:00
Rakshith R
26c07aef02 e2e: small fixes in var name and error msg
rookTolBoxPodLabel to rookToolBoxPodLabel
podPot to podOpt
fix error message for getting PV

Signed-off-by: Rakshith R <rar@redhat.com>
2021-03-25 15:15:07 +00:00
Humble Chirammal
82bc993b32 util: remove unwanted import string from module dependencies
There is no need for an extra import string when the go mod package
itself declared in the same.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-03-24 15:12:13 +00:00
Madhu Rajanna
d8f7b38d3d rbd: add exclusive-lock and journaling image features for rbd image
Current rbd plugin only supports the layering feature
for rbd image. Add exclusive-lock and journaling image
features for the rbd.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
Signed-off-by: woohhan <woohyung_han@tmax.co.kr>
2021-03-24 09:48:04 +00:00
Yati Padia
0d9548c815 Cephfs: Failed to delete snapshot
Failed to delete voluesnapshot when backend subvolume
(pvc) and ceph fs subvolume snapshot is deleted

Fixes#1647

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-03-17 10:28:08 +00:00
Yati Padia
cfb6abc067 e2e: update snapshot restore e2e to check data consistency
Currently, in rbd snapshot restore and volume clone E2E we
are not checking any data consistency after doing snapshot
restore or volume clone. Hence, this PR writes the data in
the PVC and checks the checksum of the file and verify it with
the snapshot or cloned PVC.

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-03-16 09:36:46 +00:00
Niels de Vos
c5f00a9e3d e2e: add test for the SecretsMetadataKMS provider
The new provider should be able to provision and mount an encrypted
volume.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-03-12 10:11:47 +00:00
Niels de Vos
abfe3ed9cb e2e: rework thick-provisioning test case
The stripe-size is the most efficient size to write to RBD images.
However, not all images are a multiple of stripe-size large. That means
thick-provisioning would not allocate the full image, and the process
might even fail.

This adds a 50 MB PVC to test the process, 100 MB is coincidentally a
multiple of the (default 4 MB) stripe-size, 50 MB is not.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-03-11 10:57:31 +00:00
Madhu Rajanna
a8fcf70cb8 e2e: make use of ceph users created in e2e
This commit adds support to create and delete
new ceph users to test various functionalities.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-03-03 03:44:41 +00:00
Niels de Vos
e261191f0b e2e: pass namespace once in deletePodWithLabel()
Currently framework.RunKubectl() adds `--namespace=...` 2x to the
kubectl command. Once is sufficient.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-03-02 14:44:41 +00:00
Madhu Rajanna
f4d5fdf114 e2e: add e2e for cephfs VolumeNamePrefix for PVC
when user provides an option for VolumeNamePrefix
create subvolume with the prefix which will be easy
for user to identify the subvolumes belongs to
the storageclass, Added an E2E testing to verify
that the subvolume contains the Prefix what is
provided in the storageclass.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-02-19 17:04:17 +00:00
Niels de Vos
2363d2a789 e2e: add validation for thick-provisioning
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-02-19 11:55:40 +00:00
Niels de Vos
8870575a48 e2e: compare resource.Quantity with Equals() instead of operator
When one Quantity is in GiB, and the other in Dec (bytes), the value
should be the same. However, by using ==, this is not the case. It is
needed to use Equals() for that.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-02-19 11:55:40 +00:00