Commit Graph

342 Commits

Author SHA1 Message Date
Madhu Rajanna
d943fbd265 e2e: run generic ephemeral for kubernetes 1.21+
Currently, we are skipping the generic ephemeral
testing if the kubernetes version is less than
1.21 because of this one the who test suite is
getting skipped and e2e is marked as success
in 2 minutes. This commit runs the ephemeral
tests if the kube=>1.21+. If we do this, for
the lower version we can run other tests.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-12-07 15:02:45 +00:00
Niels de Vos
ba60566cfb e2e: retry on "transport: missing content-type field" error
The e2e sometimes fail getting objects like PVCs from the Kubernetes API
server, and log the following error:

    Error getting pvc "rbd-6940" in namespace "rbd-694": rpc error: code = Unknown desc = OK: HTTP status code 200; transport: missing content-type field

By checking the error message, and initiating a retry on this failure,
CI jobs should fail less regulary.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-11-23 17:13:56 +00:00
Prasanna Kumar Kalever
0fd4db92bb e2e: add tests using different accessModes and volumeModes for rbd-nbd
Add tests for RWX and ROX accessModes for Block and FileSystem Mode
PVCs.

Fixes: #2262
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-11-23 13:13:41 +00:00
Prasanna Kumar Kalever
e5435c5bdc e2e: add deployment example for ROX used by multiple pods mounting same device
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-11-23 13:13:41 +00:00
Prasanna Kumar Kalever
fd7c8f375e cleanup: move waitForDeploymentComplete to deployment.go
Fixed the order of function parameters

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-11-23 13:13:41 +00:00
Prasanna Kumar Kalever
6e819b60e7 e2e: add helper functions to use deployment kind objects
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-11-23 13:13:41 +00:00
Prasanna Kumar Kalever
5472b66ccf e2e: start adopting to the new format of map/unmapOptions
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-11-23 08:54:37 +00:00
Humble Chirammal
929e17d21b e2e: reformat error message with consistent formatting
To make the error return consistent across e2e tests we have decided
to remove with error presence from the logs and this commit
does that for e2e/snapshot.go.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-11-22 11:07:15 +00:00
Humble Chirammal
f7f5a41774 e2e: reformat error message with consistent formatting
To make the error return consistent across e2e tests we have decided
to remove with error presence from the logs and this commit
does that for e2e/cephfs_helper.go.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-11-22 11:07:15 +00:00
Humble Chirammal
9bab088ddc e2e: reformat error message with consistent formatting
To make the error return consistent across e2e tests we have decided
to remove with error presence from the logs and this commit
does that for e2e/upgrade-rbd.go.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-11-22 11:07:15 +00:00
Humble Chirammal
fc7d3fadf5 e2e: reformat error message with consistent formatting
To make the error return consistent across e2e tests we have decided
to remove with error presence from the logs and this commit
does that for e2e/upgrade-cephfs.go.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-11-22 11:07:15 +00:00
Humble Chirammal
5ee723e634 e2e: reformat error message with consistent formatting
To make the error return consistent across e2e tests we have decided
to remove with error presence from the logs and this commit
does that for e2e/rbd_helper.go.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-11-22 11:07:15 +00:00
Humble Chirammal
4ca19ad2ff e2e: reformat error message with consistent formatting
To make the error return consistent across e2e tests we have decided
 to remove with error presence from the logs and this commit
 does that for e2e/ceph_user.go.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-11-22 11:07:15 +00:00
Humble Chirammal
14389c7b40 e2e: reformat error message with consistent formatting
To make the error return consistent across e2e tests we have decided
to remove with error presence from the logs and this commit
does that for e2e/utils.go.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-11-22 11:07:15 +00:00
Humble Chirammal
5647d4da24 e2e: reformat error message with proper error formatting
To make the error return consistent across e2e tests we have decided
to remove `with error` presence from the logs and this commit
does that for cephfs tests.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-11-22 11:07:15 +00:00
Humble Chirammal
4785c55bb8 e2e: reformat error message with proper error formatting
To make the error return consistent across e2e tests we have decided
to remove `with error` presence from the logs and this commit
does that for rbd tests.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-11-22 11:07:15 +00:00
Humble Chirammal
e6949945bb cephfs: add validation for generic ephemeral volumes
This commit adds the validation of csi cephfs driver to work with
ephemeral volume support. With ephemeral volume support a user can
specify ephemeral volumes in its pod spec and tie the lifecycle
of the PVC with the POD.

An example POD spec also included in this commit.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-11-22 06:08:23 +00:00
Humble Chirammal
df0901ddd8 rbd: add generic ephemeral volume validation
This commit adds the validation of csi RBD driver to work with
ephemeral volume support. With ephemeral volume support a user can
specify ephemeral volumes in its pod spec and tie the lifecycle
of the PVC with the POD.

An example pod spec is also included in this commit.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-11-22 06:08:23 +00:00
Humble Chirammal
aef3cc0c3c e2e: remove 1.15 based test enablement in cephfs
Considering we are far out of these release and only care about
kubernetes releases from v1.20, there is no need to have this
version check in place for the tests.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-11-18 05:04:44 +00:00
Humble Chirammal
0c5be6d12d e2e: remove 1.16 based test enablement in cephfs
Considering we are far out of these release and only care about
kubernetes releases from v1.20, there is no need to have this
version check in place for the tests.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-11-18 05:04:44 +00:00
Humble Chirammal
7090a18141 e2e: remove 1.17 based test enablement in cephfs
Considering we are far out of these release and only care about
kubernetes releases from v1.20, there is no need to have this
version check in place for the tests.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-11-18 05:04:44 +00:00
Humble Chirammal
2ac3f129c0 e2e: remove 1.15 based test enablement in rbd
considering we are far out of this release and only care about
kubernetes releases from v1.20, there is no need to have this
version check in place for the tests.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-11-18 05:04:44 +00:00
Humble Chirammal
1354cfbf03 e2e: remove 1.16 based test enablement in rbd
considering we are far out of this release and only care about
kubernetes releases from v1.20, there is no need to have this
version check in place for the tests.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-11-18 05:04:44 +00:00
Humble Chirammal
c03969fa65 e2e: remove 1.17 based test enablement in rbd
considering we are far out of this release and only care about
kubernetes releases from v1.20, there is no need to have this
version check in place for the tests.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-11-18 05:04:44 +00:00
Niels de Vos
1fa8939e84 e2e: retry when a "transport is closing" error is hit
There have been occasional CI job failures due to "transport is closing"
errors. Adding this error to the isRetryableAPIError() function should
make sure to retry the request until the connection is restored.

Fixes: #2613
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-11-17 14:07:07 +00:00
Prasanna Kumar Kalever
0bf9db822b e2e: validate encrypted image mount inside the nodeplugin
currently the mountType validation of the encrypted volume is done in
the application, we should rather validate this inside the nodeplugin
pod.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-11-16 10:12:46 +00:00
Prasanna Kumar Kalever
cee6da5313 e2e: adding io-timeout for lower kernel versions
This got removed unintentionally with
https://github.com/ceph/ceph-csi/pull/2628

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-11-16 10:12:46 +00:00
Prasanna Kumar Kalever
c97b6432e3 e2e: restrict IO with lower version kernel at rbd-nbd tests
Currently, at "perform IO on rbd-nbd volume after nodeplugin restart"
test we are performing write on the rbd-nbd based mount after nodeplugin
restart. But due to a bug in NBD driver the writes are failing, please
note NBD zero cmd timeout handling is fixed with kernel >= 5.4 and hence
we should defend on writes based on kernel version to avoid unnecessary
CI failures.

For more information see
https://github.com/ceph/ceph-csi/issues/2204#issuecomment-930941047

updates: #2204
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-11-10 16:46:50 +00:00
Humble Chirammal
de57fa1804 e2e: adjust deletion, filesystem and block tests for migration volume
this commit create and make use of migration secret in the requests and
validate various csi operations

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-10-27 18:35:00 +00:00
Rakshith R
41d894f98a e2e: add test cases for EnsureImageCleanup
This tests pvc,pvcsmartclone,snapshot deletion when
underlying images are in trash.

Signed-off-by: Rakshith R <rar@redhat.com>
2021-10-20 18:25:31 +00:00
Humble Chirammal
819f4f9048 e2e: adjust migration tests to use clusterID in the volume context
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-10-11 10:06:30 +00:00
Prasanna Kumar Kalever
a01b9821ee e2e: set rbd-nbd mounter tests cephLogStrategy to preserve
This is to preserve the rbd-nbd logs post unmap, so that the CI can dump
the available logs from logdir.

Fixes: #2451
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-10-08 14:00:42 +00:00
Madhu Rajanna
8ebc0659ab rbd: perform resize of file system for static volume
For static volume, the user will manually mounts
already existing image as a volume to the application
pods. As its a rbd Image, if the PVC is of type
fileSystem the image will be mapped, formatted
and mounted on the node,
If the user resizes the image on the ceph cluster.
User cannot not automatically resize the filesystem
created on the rbd image. Even if deletes and
recreates the kubernetes objects, the new size
will not be visible on the node.

With this changes During the NodeStageVolumeRequest
the nodeplugin will check the size of the mapped rbd
image on the node using the devicePath. and also
the rbd image size on the ceph cluster.

If the size is not matching it will do the file
system resize on the node as part of the
NodeStageVolumeRequest RPC call.

The user need to do below operation to see new size
* Resize the rbd image in ceph cluster
* Scale down all the application pods using the static
PVC.
* Make sure no application pods which are using the
static PVC is running on a node.
* Scale up all the application pods.

Validate the new size in application pod mounted
volume.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-10-06 13:15:00 +00:00
Rakshith R
f60b097f5f e2e: add testcase for thick encrypted PVC restore
Signed-off-by: Rakshith R <rar@redhat.com>
2021-10-05 07:46:57 +00:00
Rakshith R
b471cac6bd e2e: add nolint:param to retryKubectlArgs
Currently only kubectlCreate arg is used with retryKubectlArgs(),
But it maybe used later on.

Signed-off-by: Rakshith R <rar@redhat.com>
2021-10-05 07:46:57 +00:00
Rakshith R
dac4e76ae1 e2e: add testcase for PVC restore from vaultKMS to vaultTenantSAKMS
Signed-off-by: Rakshith R <rar@redhat.com>
2021-10-05 07:46:57 +00:00
Rakshith R
f63ed2ca5a e2e: modify validatePVCSnapshot() to use restoreSCName & restoreKMS
Signed-off-by: Rakshith R <rar@redhat.com>
2021-10-05 07:46:57 +00:00
Humble Chirammal
b778fe51a4 e2e: add test for migration volID detection and delete of image
This commit add test for migration delete volID detection scenario
by passing a custom volID and with the entries in configmap changed
to simulate the situation. The staticPV function also changed its
accept the annotation map which make it more general usage.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-10-04 16:06:31 +00:00
Humble Chirammal
1171111a94 e2e: deletePodWithLabel fails on unparam linter
this commit address the unparam linter error on deletePodWithLabel
function.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-10-04 16:06:31 +00:00
Humble Chirammal
a4a2dc93c1 e2e: change createCustomConfigmap to be more general
createCustomConfigmap helps to create a custom cluster entry in
the configmap, however this was coupled with subvolumegroup filling
in the cluster configuration. This commit helps to make it more
general and the subvolumegroup filling is controlled now with a flag

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-10-04 16:06:31 +00:00
Yati Padia
1cf14cd83c cleanup: rework on naming conventions
This commits replaces cephfs -> cephFS
to maintain consistency throughout the
codebase

Updates: #1465

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-09-24 06:17:17 +00:00
Humble Chirammal
4804f47b18 e2e: Add e2e for rbd migration static pvc
This commit adds e2e for rbd migration static PVCs

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-09-20 09:54:54 +00:00
Madhu Rajanna
5a53a9ad76 e2e: update e2e deployment for ceph conf
updated e2e deployment code to create and
delete the ceph conf configmap.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-09-03 14:14:43 +00:00
Rakshith R
f9d4972444 e2e: fix log msg in retryKubectlInput()
e2elog.Logf("waiting for kubectl (%s -f $q args %s) to finish", action, args)
changed to
e2elog.Logf("waiting for kubectl (%s -f args %s) to finish", action, args)

Signed-off-by: Rakshith R <rar@redhat.com>
2021-08-27 07:14:10 +00:00
Rakshith R
dbf2eb3905 e2e: ignore lines with 'Warning' in isAlreadyExistsCliError()
Signed-off-by: Rakshith R <rar@redhat.com>
2021-08-27 07:14:10 +00:00
Prasanna Kumar Kalever
55d3226d6b e2e: use io-timeout conditionally based on kernel version
We need
https://www.mail-archive.com/linux-block@vger.kernel.org/msg38060.html
inorder to use `--io-timeout=0`. This patch is part of kernel 5.4

Since minikube doesn't have a v5.4 kernel yet, lets use io-timeout value
conditionally based on kernel version at our e2e.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-08-24 17:09:09 +00:00
Prasanna Kumar Kalever
1bd2d46cdb e2e: add util to get kernel version from specified container
Currently, we get the kernel version where the e2e (client) executable runs,
not the kernel version that is used by the csi-rbdplugin pod.

Add a function that run `uname -r` command from the specified container and
returns the kernel version.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
Suggested-by: Niels de Vos <ndevos@redhat.com>
2021-08-24 17:09:09 +00:00
Prasanna Kumar Kalever
18f4a51a15 e2e: improve the debug logs for rbd-nbd
Ceph’s logging levels operate on a scale of 1 to 20, where 1 is terse
and 20 is verbose.

Format:
debug-{subsystem} = {log-level}

Setting `rbd` loglevel to 20 at our e2e tests.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-08-19 20:16:24 +00:00
Prasanna Kumar Kalever
2723353f8d e2e: add testcase for encrypted volume with rbd-nbd mounter
Fixes: #2235

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-08-11 04:10:34 +00:00
Prasanna Kumar Kalever
9669394b23 e2e: add testcase for volume expansion with rbd-nbd mounter
The rbd-nbd resize volume support with its netlink interface needs linux
kernel version >= v5.3.0

Hence define a defence check for the supported kernel version

Fixes: #2234

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-08-10 13:21:16 +00:00