Commit Graph

118 Commits

Author SHA1 Message Date
Yug
e14d649547 e2e: add cephfs test for sc, secret via helm
add cephfs test for validating storageclass and
secret deployment via helm.

Signed-off-by: Yug <yuggupta27@gmail.com>
2021-07-06 10:55:41 +00:00
Humble Chirammal
36f4c0cabb e2e: remove passing waitgroups for go routines
We are unwantedly passing around waitgroups which is not required.
This address the same inside e2e/cephfs.go and remove waitgroup arg

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-06-29 15:07:56 +00:00
Humble Chirammal
49d11bf5ea e2e: reformat long lines in this package to 120 chars
We have many declarations and invocations..etc with long lines which are
very difficult to follow while doing code reading. This address the issues
in 'e2e/cephfs*.go' files to restrict the line length to 120 chars.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-06-28 14:43:49 +00:00
Humble Chirammal
f3d400730e e2e: make use of snapshot v1 api in cephfs.go
The import name of snapshot api package has also been changed
from `vs` to `snapapi` for consistency.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-06-24 17:35:07 +00:00
Humble Chirammal
4302e5ee67 cleanup: correct createORDeleteCephfsResources() function name
Along with correcing the name of the function, other typos are also
addressed

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-06-11 08:04:07 +00:00
Madhu Rajanna
fba6a2d0c3 deploy: add csidriver object for cephfs and rbd
csidriver object can be created on the kubernetes
for below reason.

If a CSI driver creates a CSIDriver object,
Kubernetes users can easily discover the CSI
Drivers installed on their cluster
(simply by issuing kubectl get CSIDriver)

Ref: https://kubernetes-csi.github.io/docs/csi-driver-object.html#what-is-the-csidriver-object

attachRequired is always required to be set to
true to avoid issue on RWO PVC.

more details about it at https://github.com/rook/rook/pull/4332

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-03-31 13:41:35 +00:00
Rakshith R
9e957520ae e2e: add test case for app deletion with unmounted cephFS volume
Checks app deletion when cephFS volume is already unmounted.
Creates app, pvc and binds them. Unmounts the volume through
umount cmd in cephFS plugin and checks app deletion.

Signed-off-by: Rakshith R <rar@redhat.com>
2021-03-25 15:15:07 +00:00
Yati Padia
0d9548c815 Cephfs: Failed to delete snapshot
Failed to delete voluesnapshot when backend subvolume
(pvc) and ceph fs subvolume snapshot is deleted

Fixes#1647

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-03-17 10:28:08 +00:00
Yati Padia
cfb6abc067 e2e: update snapshot restore e2e to check data consistency
Currently, in rbd snapshot restore and volume clone E2E we
are not checking any data consistency after doing snapshot
restore or volume clone. Hence, this PR writes the data in
the PVC and checks the checksum of the file and verify it with
the snapshot or cloned PVC.

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-03-16 09:36:46 +00:00
Madhu Rajanna
a8fcf70cb8 e2e: make use of ceph users created in e2e
This commit adds support to create and delete
new ceph users to test various functionalities.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-03-03 03:44:41 +00:00
Madhu Rajanna
f4d5fdf114 e2e: add e2e for cephfs VolumeNamePrefix for PVC
when user provides an option for VolumeNamePrefix
create subvolume with the prefix which will be easy
for user to identify the subvolumes belongs to
the storageclass, Added an E2E testing to verify
that the subvolume contains the Prefix what is
provided in the storageclass.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-02-19 17:04:17 +00:00
Mudit Agarwal
4eb5bd6ba6 e2e: add e2e for snapshot retention fetaure.
Added e2e for snapshot retention feature.

Signed-off-by: Mudit Agarwal <muagarwa@redhat.com>
2021-02-19 10:03:55 +00:00
Madhu Rajanna
1aaccd0e5a e2e: add validation for subvolumePath
as we are adding the subvolumepath to the
PV spec. this commits adds an E2E testing
to verify it.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-02-11 11:31:22 +00:00
Niels de Vos
4abe128bd8 e2e: log events from deployment namespace
When tests run and something goes wrong during deployment, not all
information is available. Logging the events from the namespace where
Ceph-CSI (and Vault) is deployed, might help with troubleshooting.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-17 11:40:57 +00:00
Niels de Vos
96eafecad8 e2e: do not use Failf() to abort tests in a go-routine
There are several go-routines where Failf() is called, which will cause
a Golang panic inside the Ginko test framework. Instead of aborting the
go-routine, capture the error and check for failures once all
go-routines have finished.

Updates: #1359
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-05 11:26:20 +00:00
Humble Chirammal
b4452f0787 e2e: validate the error return instead of object for consitency
If loadPVC() fails, it return error and we expect the PVC object
to be nil too. In many places we check on the error and exit.
However in few places we are looking at PVC object.
This commit make the condition check on `err` instead of `PVC`
object for consistency.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-10-21 13:56:12 +00:00
Niels de Vos
cfddc9f887 e2e: use k8sVersionGreaterEquals() for cephfs tests
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-19 16:00:39 +00:00
Nico Berlee
6a5f1380b0 deploy: add pod anti-affinity for provisioner deployments
The added anti-affinity rules prevent provisioner operators from scheduling on
the same nodes. The kubernetes scheduler will spread the pods across nodes to
improve availability during node failures.

Signed-off-by: Nico Berlee <nico.berlee@on2it.net>
2020-09-29 09:29:58 +00:00
Niels de Vos
f41cd105c4 e2e: add simple ceph-fuse StorageClass tests
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-09-23 12:12:51 +00:00
Niels de Vos
85df59531b e2e: pass SC parameters to createCephfsStorageClass()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-09-23 12:12:51 +00:00
Madhu Rajanna
be3a78afaf e2e: update cephfs ROX test case
updated cephfs ROX test case to create
ROX Clone PVC instead of creating an empty PVC

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-09-10 13:47:23 +00:00
Madhu Rajanna
b4693dcffe e2e: rework on E2E framework
rework of E2E framework for better code
organization and add more helpful logs for
debugging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-09-10 07:40:13 +00:00
Madhu Rajanna
b5320d9273 e2e: add backend validation for pvc restore and clone
added backend subvolume validation for cephfs
snapshot restore and pvc to pvc clone.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-11 16:21:29 +00:00
Madhu Rajanna
2307687dc2 e2e: add backend validation of cephfs pvc
added a backend validation to check subvolumes
count when we create and delete pvc.

fixes #1316

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-11 16:21:29 +00:00
Humble Chirammal
ef1785ce4d e2e: remove volume populate test case and also change writeDataInPod()
As we are populating the volume in other two test cases for clone and
snapshot operation, we dont need a specific test case now.
WriteDataInPod() function is also changed to take the pod spec and write
some data to it

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-08 00:11:35 +00:00
Humble Chirammal
074feddaf3 e2e: add test cases for cephfs snapshot creation,deletion and restore
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-08 00:11:35 +00:00
Yug
9b30969594 cephfs: add upgrade testing
Upgrade testing will enable us to keep
in check the backward compatibility of
earlier releases.

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-07-26 03:41:53 +00:00
Madhu Rajanna
b773e821c8 ci: use testCephFS flag to skip cephfs tests
This commit fixes the flag check to skip
cephfs E2E tests.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-09 13:50:21 +00:00
Madhu Rajanna
63b795d165 e2e: provide an option to skip rbd and cephfs e2e
Added two new parameters for e2e test to skip
rbd and cephfs tests. This will help us to
run more test in Travis CI.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-09 06:42:28 +00:00
Madhu Rajanna
21bd26ebc4 e2e: Add E2E for ROX PVC
Added E2E testing for creation
and mounting of ROX PVC, if the
PVC accessmode is ReadOnlyMany
the application pod should not get
write access to it.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-06-30 17:43:41 +00:00
Madhu Rajanna
37979da5b2 e2e: Add E2E for app readonly options
If the mount option is readonly in app
pod, the pod should not get the write
access to the mounted cephfs subvolume.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-06-30 17:43:41 +00:00
Yug Gupta
c9ad904331 cephfs: Add support for multiple subvolumegroups
With the current code base, the subvolumegroup will
be created once, and even for a different cluster,
subvolumegroup creation is not allowed again.

Added support multiple subvolumegroups creation by
validating one subvolumegroup creation per cluster.

Fixes: #1123

Signed-off-by: Yug Gupta <ygupta@redhat.com>
2020-06-26 09:19:24 +00:00
Mudit Agarwal
fb1e689648 test: add e2e test for cephFS static PV
With the change in https://github.com/ceph/ceph-csi/issues/382, support for static PV for CephFS was added.
This change is to add e2e test case for the same to ensure that we do not regress the functionality.

Issue: https://github.com/ceph/ceph-csi/issues/669

Signed-off-by: Mudit Agarwal <muagarwa@redhat.com>
2020-05-18 08:30:50 +00:00
ShyamsundarR
e73921f268 Add e2e tests for topology based provisioning
- This commit adds tests only for RBD, as CephFS still needs
an enhancement in CephFS subvolume commands to effectively use
topology based provisioning

Signed-off-by: ShyamsundarR <srangana@redhat.com>
2020-04-14 14:14:29 +00:00
Humble Chirammal
4c96ad3c85 Changes to e2e to accomodate client-go changes and RunKubectlInput
With client-go v1.18.0 there is a change where Signatures on methods
in generated clientsets, dynamic, metadata, and scale clients have been
modified to accept context.Context as a first argument.
Signatures of Create, Update, and Patch methods have been updated to accept
CreateOptions, UpdateOptions and PatchOptions respectively.
Signatures of Delete and DeleteCollection methods now accept DeleteOptions
by value instead of by reference

The framework.RunkubectlInput now accepts namespace as the first parameter
which is also accommodated with this PR.

Signed-off-by: Humble Chirammal hchiramm@redhat.com
2020-04-14 10:50:12 +00:00
Madhu Rajanna
d09ffbd6de helm: add helm charts E2E
This PR adds the support for helm
installation, and cephcsi helm charts
deployment and teardown and also runs E2E
on for helm charts.

Add socat to provide port forwadring access for helm

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-04-06 11:01:25 +00:00
Madhu Rajanna
e788328750 Refractor E2E to reduce code duplication
Updated E2E to reduce code duplication
and create resouces in different namespaces.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-03-20 10:16:23 +00:00
Madhu Rajanna
c45c426215 Add cephcsi namespace and rook namespace flag
Added namespace flag to cephcsi to deploy cephcsi
resouces in different namespace.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-03-20 10:16:23 +00:00
Madhu Rajanna
59fe8c1f2f Provide option to skip ceph-csi plugin deployment
To test helm charts in CI we need to skip the ceph-csi
deployment in E2E, This PR provides an option in E2E
to enable/disable cephcsi deployment.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-03-20 10:16:23 +00:00
Madhu Rajanna
8dcb6a6105 Handle Delete operation if pool not found
If the backend rbd or cephfs pool is already deleted
we need to return success to the  DeleteVolume RPC
call to make it idempotent.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-02-11 15:58:15 +00:00
ShyamsundarR
b93ed21fe8 Add e2e test case for DeleteVolume success on backend volume deletion
If a backend volume is deleted, DeleteVolume call for the same should
succeed, detecting the image is missing and delete the related OMaps.

This commit adds a test case to ensure this is occuring correctly.

Updates #474

Signed-off-by: ShyamsundarR <srangana@redhat.com>
2020-01-29 10:05:13 +00:00
Oguz Kilcan
aadce54b2f Added PodSecurityPolicy support 2020-01-22 08:19:42 +00:00
wilmardo
f04af5742d refact: Remove Kubernetes 1.13.x support
Signed-off-by: wilmardo <info@wilmardenouden.nl>
2020-01-20 10:32:30 +00:00
Humble Chirammal
2d0ed298e3 Update framework imports and its functions.
Signed-off-by: Cedric Hauber <hauber.c@gmail.com>
2020-01-17 12:06:02 +00:00
Madhu Rajanna
7d15992769 Enable logging in E2E if test fails
source: https://github.com/kubernetes/kubernetes/blob/master/test/e2e/framework/kubectl/kubectl_utils.go
kubectlLogPod function is not exposed in above code so copied it.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-12-18 12:35:17 +00:00
Woohyung Han
5c96ab1d0e Fix to use kubectl create not kubectl apply in the e2e
Signed-off-by: Woohyung Han <techhanx@gmail.com>
2019-12-11 09:31:03 +00:00
Woohyung Han
2c9d711463 Move rook-deploy code from e2e to ./scripts/minikube.sh
We have the e2e test with --deploy-rook=true that makes all test
environment. It works fine, but It does not seem to be the role of
e2e test. In addition, when developing the code we need to run full
test scenario with deploying rook every time, or we need to build
rook environment by hand. Move rook-deploy code to minikube.sh.
2019-12-05 12:32:59 +00:00
Woohyung Han
685e2540a8 Fix typo in deleteConfigMap 2019-12-04 04:31:50 +00:00
Humble Chirammal
ac09c5553c Add E2E for cephfs resize functionality
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2019-11-27 14:00:47 +00:00
Madhu Rajanna
64ca401a51 Fix mon endpoint issue in E2E
in toolbox mon endpoints are not
updated properly, this is causing an issue in E2E
this PR is a workaround to fix this issue.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-09-04 11:12:07 +00:00
Madhu Rajanna
ae67534a44 Add wait logic to check fsid from toolbox
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-08-30 06:50:32 +00:00
Madhu Rajanna
4ed187eb3f Fix issues in E2E cleanup
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-08-07 20:26:20 +05:30
Daniel-Pivonka
0063727199 Make parameter pool optional in CephFS storageclass
Signed-off-by: Daniel-Pivonka <dpivonka@redhat.com>
2019-08-07 13:30:38 +00:00
Madhu Rajanna
02bcb5f16a Enable leader election in v1.14+
Use Deployment with leader election instead of StatefulSet

Deployment behaves better when a node gets disconnected
from the rest of the cluster - new provisioner leader
is elected in ~15 seconds, while it may take up to
5 minutes for StatefulSet to start a new replica.

Refer: kubernetes-csi/external-provisioner@52d1fbc

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-08-05 07:11:44 +00:00
Poornima G
da111de65d Revert "Temperory workaround for the regression to pass"
The rook commit e52efdb8d4ed98129708d49cbbbbbcb083b02d6c fixes the
issue. Hence reverting this patch.
2019-07-25 08:49:43 +00:00
Madhu Rajanna
f4c80dec9a Implement NodeStage and NodeUnstage for rbd
in NodeStage RPC call  we  have to map the
device to the node plugin and make  sure  the
the device will be mounted to  the global path

in  nodeUnstage request unmount the device from
global path and unmap the device

if the volume mode is block  we will be creating
a file inside a stageTargetPath  and it will be
considered  as the global path

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-07-24 12:49:21 +00:00
Poornima G
7d8f465746 Temperory workaround for the regression to pass
Currently the cephfs PVC create/delete and all other operations
related to cephfs are failing. One of the recent commits in rook
900abbc967e108ad622648b740a7c57f1268209f has modified ceph-mgr
to run as ceph user rather than root user. The ceph user currently
has no permission to write to the root of the cephfs filesystem.
The fix will be external to CSI itself, but until that lands, sending
a workaround patch so the CSI CI is unblocked

In this patch, we are setting the permission 777 on root of the cephfs
filesystem. Thus ceph user will be able to modify the cephfs filesystem.

Signed-off-by: Poornima G <pgurusid@redhat.com>
2019-07-24 12:04:46 +00:00
Humble Chirammal
3bc6771df8 Migrate from framwork.Logf and also use new nsenter interface
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2019-06-24 13:08:29 +00:00
Madhu Rajanna
2e1fc352b3 Add validation of backend rbd image and snapshots
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-06-23 11:53:37 +00:00
Madhu Rajanna
1a2d71e580 Add code to test snapshot,pvc clone and pvc block mode
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-06-23 11:53:37 +00:00
Madhu Rajanna
983f28ad2f Revert "Use Deployment with leader election instead of StatefulSet"
This reverts commit a151bec94b.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-06-14 13:39:03 +00:00
Madhu Rajanna
69662e63ed change permission of targetpath
setting the permission of  targetpath to 777
will allow non-root user to write to pv.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-06-12 14:24:37 +00:00
Humble Devassy Chirammal
46b19a93f5
Merge pull request #416 from Madhu-1/update-golanglint
Enable all static-checks in golangci-lint
2019-06-10 16:49:40 +05:30
Madhu Rajanna
a38986fce0 Enable all static-checks in golangci-lint
* Enable all static-checks in golangci-lint
* Update golangci-lint version
* Fix issue found in golangci-lint

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-06-10 15:56:17 +05:30
Humble Chirammal
66bb0e4e54 Combine the variable section in e2e and also introuce new vars.
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2019-06-10 13:50:30 +05:30
Madhu Rajanna
a151bec94b Use Deployment with leader election instead of StatefulSet
Deployment behaves better when a node gets disconnected from the rest of
the cluster - new provisioner leader is elected in ~15 seconds, while
it may take up to 5 minutes for StatefulSet to start a new replica.

Refer: 52d1fbcf9d

Fixes: #335

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-06-10 09:51:22 +05:30
Madhu Rajanna
a4655f7549 create config map for cephfs
add fs name to storageclass in e2e

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-06-07 15:38:23 +05:30
Madhu Rajanna
fc529f4831 Add E2E framework
deploys rook
deploy cephfs plugin
deploy rbd plugin
create PVC/POD to test cephfs
create PVC/POD to test rbd

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-06-04 11:39:42 +05:30