Commit Graph

121 Commits

Author SHA1 Message Date
Madhu Rajanna
8ab04d4160 e2e: add backend image validation for rbd
validate backend rbd images count in each
E2E test cases. This helps a lot to catch
the issues in each test case.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit d5c19954c1)
2020-08-24 09:24:09 +00:00
Mudit Agarwal
46acd6b2cd doc: update e2e README for snapshot operation
Updated e2e README with the prerequisites to run snapshot/clone e2e tests.

Signed-off-by: Mudit Agarwal <muagarwa@redhat.com>
2020-08-12 16:27:49 +00:00
Mehdy Khoshnoody
204487383d e2e: Add e2e test cases for rbd rados namespace
These test cases are will be executed against a rados namespace.
- Create a PVC and bind it to an app.
- Resize block PVC and check device size.
- Create a PVC clone and bind it to an app.

Signed-off-by: Mehdy Khoshnoody <mehdy.khoshnoody@gmail.com>
2020-08-12 16:22:58 +05:30
Madhu Rajanna
b5320d9273 e2e: add backend validation for pvc restore and clone
added backend subvolume validation for cephfs
snapshot restore and pvc to pvc clone.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-11 16:21:29 +00:00
Madhu Rajanna
2307687dc2 e2e: add backend validation of cephfs pvc
added a backend validation to check subvolumes
count when we create and delete pvc.

fixes #1316

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-11 16:21:29 +00:00
Madhu Rajanna
141b89f514 e2e: add function to get subvolume list
added a helper function in e2e to list subvolumes
in a filesystem

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-11 16:21:29 +00:00
Humble Chirammal
ef1785ce4d e2e: remove volume populate test case and also change writeDataInPod()
As we are populating the volume in other two test cases for clone and
snapshot operation, we dont need a specific test case now.
WriteDataInPod() function is also changed to take the pod spec and write
some data to it

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-08 00:11:35 +00:00
Humble Chirammal
074feddaf3 e2e: add test cases for cephfs snapshot creation,deletion and restore
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-08 00:11:35 +00:00
Humble Chirammal
90762342a0 e2e: introduce createCephFSSnapshotClass creation
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-08 00:11:35 +00:00
Humble Chirammal
2d72550a39 e2e: introduce writeDataInPod() to write data to an attached PVC
writeDataInPod() write data to the attached PVC using `dd` command
It leave the pod and pvc state as it is.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-08 00:11:35 +00:00
Yug
02b4a7175c rbd: add upgrade testing
Upgrade testing will enable us to keep
in check the backward compatibility of
earlier releases.

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-07-26 03:41:53 +00:00
Yug
9b30969594 cephfs: add upgrade testing
Upgrade testing will enable us to keep
in check the backward compatibility of
earlier releases.

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-07-26 03:41:53 +00:00
Yug
9c0d5abb5a doc: Add README for upgrade-testing
Update README with upgrade testing parameters.

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-07-26 03:41:53 +00:00
Madhu Rajanna
5168ad7ddf e2e: create/delete snap and clone in parallel
In rbd E2E testing,we need to create snap and clone
as parallel operation.

This helps us to insure that functionality works when
we have parallel delete and create operations and also
it helps to catch bugs when we get parallel requests.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-21 13:25:19 +00:00
Yug
e73fe64a0d cleanup: address gosec warnings
gosec warns about security problems by scanning the
Go AST.

Issues Reported:
G101 (CWE-798): Potential hardcoded credentials (Confidence: LOW, Severity: HIGH)
G204 (CWE-78): Subprocess launched with variable (Confidence: HIGH, Severity: MEDIUM)
G304 (CWE-22): Potential file inclusion via variable (Confidence: HIGH, Severity: MEDIUM)

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-07-21 08:36:24 +00:00
Yug
628ae9e982 cleanup: use wrapped static errors instead of dynamic
In Go 1.13, the fmt.Errorf function supports a new %w verb.
When this verb is present, the error returned by fmt.Errorf
will have an Unwrap method returning the argument of %w,
which must be an error. In all other ways, %w is identical to %v.

Updates: #1227

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-07-21 08:36:24 +00:00
Yug
2bfe7670d2 cleanup: address govet warnings
We had "ns" as a parameter and then trying to
declare it also as a local variable, which is what
the complaint about "shadowing" refers to.

Issue reported:
shadow: declaration of "ns" shadows declaration at line 57 (govet)

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-07-21 08:36:24 +00:00
Yug
7f94a57908 cleanup: address godot warnings
Top level comments should end in a period

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-07-21 08:36:24 +00:00
Madhu Rajanna
9077c25c15 e2e: add e2e to test pvc-pvc cloning
Added an e2e testcase to test pvc-pvc
cloning.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-10 14:02:12 +00:00
Madhu Rajanna
b773e821c8 ci: use testCephFS flag to skip cephfs tests
This commit fixes the flag check to skip
cephfs E2E tests.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-09 13:50:21 +00:00
Madhu Rajanna
63b795d165 e2e: provide an option to skip rbd and cephfs e2e
Added two new parameters for e2e test to skip
rbd and cephfs tests. This will help us to
run more test in Travis CI.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-09 06:42:28 +00:00
Madhu Rajanna
6838d30aff e2e: reduce the pod cound and wait time
Reduced the number of pods created
in ROX E2E to save some time in E2E
and changed the waiting time from 2 to 1
min.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-06 10:08:31 +00:00
Niels de Vos
c9d5bc8a08 cleanup: use errors.As() in e2e/utils
See-also: https://github.com/golang/go/wiki/ErrorValueFAQ#how-should-i-change-my-error-handling-code-to-work-with-the-new-features
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-03 09:12:48 +00:00
Madhu Rajanna
47fb5f2299 e2e: update E2E testing for snapshot
with new implemntation when user creates a snapshot
in backend we are creating rbd image, we need to
validate the total images count in backend when
creating snapshots and clones.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-01 08:21:47 +00:00
Madhu Rajanna
21bd26ebc4 e2e: Add E2E for ROX PVC
Added E2E testing for creation
and mounting of ROX PVC, if the
PVC accessmode is ReadOnlyMany
the application pod should not get
write access to it.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-06-30 17:43:41 +00:00
Madhu Rajanna
37979da5b2 e2e: Add E2E for app readonly options
If the mount option is readonly in app
pod, the pod should not get the write
access to the mounted cephfs subvolume.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-06-30 17:43:41 +00:00
Yug Gupta
c9ad904331 cephfs: Add support for multiple subvolumegroups
With the current code base, the subvolumegroup will
be created once, and even for a different cluster,
subvolumegroup creation is not allowed again.

Added support multiple subvolumegroups creation by
validating one subvolumegroup creation per cluster.

Fixes: #1123

Signed-off-by: Yug Gupta <ygupta@redhat.com>
2020-06-26 09:19:24 +00:00
Madhu Rajanna
39561b9675 e2e: Add E2E to mount rbd PVC as ro in app
Added an E2E to mount rbd PVC as readonly
in application pod and try to create some
file in Readonly PVC,when we try to create
files on RO PVC, we should get error.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-06-22 06:15:40 +00:00
Madhu Rajanna
649aeb7aaf rbd: Add support for rbd ROX PVC mounting
if the PVC access mode is ReadOnlyMany
or single node readonly, mounting the rbd
device path to the staging path as readonly
to avoid the write operation.

If the PVC acccess mode is readonly, mapping
rbd images as readonly.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-06-22 06:15:40 +00:00
Humble Chirammal
dcbf71dd5d cleanup: avoid double import of kubernetes library in e2e utils
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-06-19 08:42:08 +00:00
Humble Chirammal
323cc0e3bb cleanup: avoid comparing errors directly
Go 1.13 contains support for error wrapping. To support wrapping,
fmt.Errorf now has a %w verb for creating wrapped errors, and three
new functions in the errors package ( errors.Unwrap, errors.Is and
errors.As) simplify unwrapping and inspecting wrapped errors.

With this change, If we currently compare errors using ==, we have to
use errors.Is instead. Example:

if err == io.ErrUnexpectedEOF

becomes

if errors.Is(err, io.ErrUnexpectedEOF)

https://tip.golang.org/doc/go1.13#error_wrapping

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-06-17 06:26:11 +00:00
Humble Chirammal
c1a269191c doc: make sure configmap object referred or documented correctly
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-06-10 11:33:10 +00:00
Madhu Rajanna
44da7ffb4e e2e: Add testcase of invalid mount option
Added a test case to cover unmap of rbd image
if the mounting fails. if we pass the invalid
mount option the expectation is that mounting
of rbd image to stagingpath fails. as the unmap
happens it should not block the rbd pvc deletion
saying rbd image is in-use.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-06-01 11:27:25 +00:00
Madhu Rajanna
78cdf9d4db e2e: Add option to set mountoptions in sc
Updated E2E code to pass mountoptions when
creating a storageclass.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-06-01 11:27:25 +00:00
Yug Gupta
753f6b7fec cleanup: fix golint warnings in util, e2e
util: golint warns about exported methods to have a
comment or to unexport them.

e2e: golint warns about package comment to be of the form
"Package e2e ..."

Reported-by: https://goreportcard.com/report/github.com/ceph/ceph-csi

Updates: #975

Signed-off-by: Yug Gupta <ygupta@redhat.com>
2020-05-20 13:18:41 +00:00
Madhu Rajanna
b0270ced41 e2e: reword sentences in e2e
as kube is the shortform for kubernetes.
its expected to mention full form kubernetes
in the e2e tests.

Updated few wordings in the e2e.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-05-20 06:37:36 +00:00
Madhu Rajanna
7a5e064f50 e2e: Add a new function to run commands in toolboxpod
Updated code to use new function execCommandInToolBoxPod
to run commands in toolboxpod.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-05-20 06:37:36 +00:00
Madhu Rajanna
0f6c6c15a8 e2e: remove explict setting of kind and apiversion
we dont need to explictly set the kind
and apiversion in the snapshot class object
as it is already set in snapshotclass.yaml file

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-05-20 06:37:36 +00:00
Mudit Agarwal
fb1e689648 test: add e2e test for cephFS static PV
With the change in https://github.com/ceph/ceph-csi/issues/382, support for static PV for CephFS was added.
This change is to add e2e test case for the same to ensure that we do not regress the functionality.

Issue: https://github.com/ceph/ceph-csi/issues/669

Signed-off-by: Mudit Agarwal <muagarwa@redhat.com>
2020-05-18 08:30:50 +00:00
Madhu Rajanna
f0b3cee94a e2e: define replicapool as constant
the static check is failing as the replicapool
is used in 3 or more places, we need to define
a variable and use it.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-05-15 08:19:32 +00:00
Madhu Rajanna
76a55350aa e2e: Pass context.TODO() to kubernetes snapshot client
in external-snapshotter the client has changed we
need to pass the context to snapshot api calls.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-05-15 08:19:32 +00:00
Madhu Rajanna
1f13692000 ci: skip snapshot E2E if kube<1.17+
snapshot beta CRD wont work if the
kubernetes version is less than 1.17.0
as the snapshot CRD wont be installed
we cannot test the snapshot,so disabling
it if the kube version is less than 1.17

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-05-15 08:19:32 +00:00
Madhu Rajanna
c5bf95b625 e2e: Enable E2E for rbd snapshot
Updated E2E to use beta client and beta
Snapshot resources.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-05-15 08:19:32 +00:00
Yug Gupta
2cdf5c3b9f util: make util.ClusterInfo usable outside util package
functions like getClusterInfo() returns struct
instead of a set of strings.

Fix: #998

Signed-off-by: Yug Gupta <ygupta@redhat.com>
2020-05-14 19:43:21 +00:00
Mathias Merscher
0991cdf498 make CephFS SubvolumeGroup configurable
The name of the CephFS SubvolumeGroup for the CSI volumes was hardcoded to "csi". To make permission management in multi tenancy environments easier, this commit makes it possible to configure the CSI SubvolumeGroup.

related to #798 and #931
2020-05-04 05:50:06 +00:00
Niels de Vos
7d642b791c e2e: fix IneffAssign warnings in checkPVSelectorValuesForPVC()
IneffAssign warns about the two following statements:

  Line 1342: warning: ineffectual assignment to rFound (ineffassign)
  Line 1350: warning: ineffectual assignment to zFound (ineffassign)

rFound and zFound should be set before entering the loop, otherwise the
initial value will overwrite the updated value on each iteration.

Reported-by: https://goreportcard.com/report/github.com/ceph/ceph-csi
Updates: #975
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-04-24 08:35:36 +00:00
ShyamsundarR
1a8f8e3c24 Add support for erasure coded pools
This commit adds support to mention dataPool parameter for the
topology constrained pools in the StorageClass, that can be
leveraged to mention erasure coded pool names to use for RBD
data instead of the replica pools.

Signed-off-by: ShyamsundarR <srangana@redhat.com>
2020-04-14 14:14:29 +00:00
ShyamsundarR
e73921f268 Add e2e tests for topology based provisioning
- This commit adds tests only for RBD, as CephFS still needs
an enhancement in CephFS subvolume commands to effectively use
topology based provisioning

Signed-off-by: ShyamsundarR <srangana@redhat.com>
2020-04-14 14:14:29 +00:00
ShyamsundarR
5c4abf8347 Add topology support to ceph-csi
Signed-off-by: ShyamsundarR <srangana@redhat.com>
2020-04-14 14:14:29 +00:00
Humble Chirammal
4c96ad3c85 Changes to e2e to accomodate client-go changes and RunKubectlInput
With client-go v1.18.0 there is a change where Signatures on methods
in generated clientsets, dynamic, metadata, and scale clients have been
modified to accept context.Context as a first argument.
Signatures of Create, Update, and Patch methods have been updated to accept
CreateOptions, UpdateOptions and PatchOptions respectively.
Signatures of Delete and DeleteCollection methods now accept DeleteOptions
by value instead of by reference

The framework.RunkubectlInput now accepts namespace as the first parameter
which is also accommodated with this PR.

Signed-off-by: Humble Chirammal hchiramm@redhat.com
2020-04-14 10:50:12 +00:00