Commit Graph

413 Commits

Author SHA1 Message Date
Yati Padia
bfda5fa57f cleanup: resolve revive linter issue
revive linter checks for var-declaration
format.
For example:
"e2e/rbd_helper.go:441:36: var-declaration:
should drop = nil from declaration of
var noPVCValidation; it is the zero value (revive)
var noPVCValidation validateFunc = nil"

Updates: #1586

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-07-19 08:39:32 +00:00
Prasanna Kumar Kalever
10fc639d68 ci: fix nolintlint warnings
warnings from golangci-lint:

e2e/pod.go:207:122: directive `//nolint:unparam,lll // cn can be used
with different inputs later` is unused for linter unparam (nolintlint)
func execCommandInContainer(f *framework.Framework, c, ns, cn string,
opt *metav1.ListOptions) (string, string, error) { //nolint:unparam,lll
// cn can be used with different inputs later

e2e/pod.go:307:70: directive `//nolint:unparam // skipNotFound can be
used with different inputs later` is unused for linter unparam (nolintlint)
func deletePodWithLabel(label, ns string, skipNotFound bool) error {
//nolint:unparam // skipNotFound can be used with different inputs later

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-07-16 16:30:58 +00:00
Prasanna Kumar Kalever
fd3bf1750b e2e: fix the testcases for rbd-nbd
Now that the healer functionaity for mounter processes is available,
lets start, using it.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-07-16 16:30:58 +00:00
Niels de Vos
4d4a2a7814 e2e: prevent re-using empty pvc object
When an error occurs, the pvc object is overwritten in the
PollImmediate() loop. Re-using the pvc.Namespace results in error
messages like

    Error getting pvc in namespace: '': an empty namespace may not be set when a resource name is provided

and prevents the retry by PollImmediate() to never succeed. Storing the
namespace in a local variable prevents this from happening.

Reported-by: Rakshith R <rar@redhat.com>
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-07-14 10:18:51 +00:00
Niels de Vos
f7ae33c67c e2e: only call error check functions when err != nil
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-07-14 10:18:51 +00:00
Niels de Vos
075a4087d7 e2e: mark "etcdserver: request timed out" errors as retryable
There are regular CI failures where etcdserver times out. These errors
seem not to get caught by any of the existing error comparing. Matching
the error by string should prevent temporary etcdserver issues now too.

Updates: #2218
Closes: #1969
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-07-14 10:18:51 +00:00
Niels de Vos
e3c7dea7d6 e2e: add test for Vault with ServiceAccount per Tenant
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-07-13 17:16:35 +00:00
Yati Padia
4a649fe17f cleanup: resolve godot linter
This commit resolves godot linter issue
which says "Comment should end in a period (godot)".

Updates: #1586

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-07-13 06:50:03 +00:00
Niels de Vos
855f9080d1 e2e: refactor KMS configuration and usage
This adds a new `kmsConfig` interface that can be used to validate
different KMS services and setting. It makes checking for the available
support easier, and fetching the passphrase simpler.

The basicKMS mirrors the current validation of the KMS implementations
that use secrets and metadata. vaultKMS can be used to validate the
passphrase stored in a Vault service.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-07-12 14:57:39 +00:00
Yati Padia
84c1fe52c7 cleanup: resolve exhaustive linter
This commit resolves exhaustive linter
error.

Updates: #2240

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-07-12 04:47:08 +00:00
Rakshith R
b27d6319ca e2e: add e2e for user secret based metadata encryption
This commit adds e2e for user secret based metadata encryption,
adds user-secret.yaml and makes required changes in kms-connection-details,
kms-config yamls.

Signed-off-by: Rakshith R <rar@redhat.com>
2021-07-08 17:06:02 +00:00
Yati Padia
ffab37f44f cleanup: resolves gocritic linter issues
This commit resolves gocritic linter errors.

Updates: #2250

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-07-08 05:19:26 +00:00
Madhu Rajanna
f471a206c5 e2e: update e2e to validate thick metadata
updated e2e tests to validate the thick
metadata set on the rbd image for thick PVC.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-07-07 08:31:10 +00:00
Yug
e14d649547 e2e: add cephfs test for sc, secret via helm
add cephfs test for validating storageclass and
secret deployment via helm.

Signed-off-by: Yug <yuggupta27@gmail.com>
2021-07-06 10:55:41 +00:00
Yug
7d28896e36 e2e: add rbd test for testing sc, secret via helm
add rbd test for validating storageclass and
secret deployment via helm.

Signed-off-by: Yug <yuggupta27@gmail.com>
2021-07-06 10:55:41 +00:00
Rakshith R
fb365005da e2e: update e2epv.WaitOnPVandPVC() to match new parameters
Added &framework.TimeoutContext{ClaimBound: timeout, PVBound: timeout}
to parameters of function call in updated package.

Signed-off-by: Rakshith R <rar@redhat.com>
2021-07-01 03:35:23 +00:00
Madhu Rajanna
3e9172ab31 e2e: check isRetryableAPIError during polling
check isRetryableAPIError error when the
function is polling for the resouce.

updates: #1969

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-06-30 10:00:44 +00:00
Yati Padia
3c773b24e5 e2e: validate encryption keys in KMS
this commit is to validate if the encrypted
keys are created and deleted properly while
pvc-pvc clone images

Updates: #2022

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-06-30 08:21:01 +00:00
Humble Chirammal
36f4c0cabb e2e: remove passing waitgroups for go routines
We are unwantedly passing around waitgroups which is not required.
This address the same inside e2e/cephfs.go and remove waitgroup arg

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-06-29 15:07:56 +00:00
Humble Chirammal
4657ac7f00 e2e: remove passing waitgroups for go routines
We are unwantedly passing around waitgroups which is not required.
This address the same inside e2e/utils.go and remove waitgroup arg

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-06-29 15:07:56 +00:00
Humble Chirammal
053f7ee213 e2e: remove passing waitgroups reference for go routines
We are unwantedly passing around waitgroups which is not required.
This address the same inside e2e/rbd_helper.go and remove the waitgroup
arg

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-06-29 15:07:56 +00:00
Yati Padia
7f7a41d23a cleanup: use %w to format errors
Use %w instead of %v to format errors.

Updates: #1586

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-06-29 13:05:31 +00:00
Yati Padia
c851b69160 cleanup: error is not nil but returns nil
In few places the error is not nil still it
returns nil.

Updates: #1586

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-06-28 17:54:38 +00:00
Humble Chirammal
c2cda4909b e2e: reformat long lines in this package to 120 chars
We have many declarations and invocations..etc with long lines which are
very difficult to follow while doing code reading. This address the issues
in 'e2e/upgrade_*.go' files to restrict the line length to 120 chars.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-06-28 14:43:49 +00:00
Humble Chirammal
41ecaadc82 e2e: reformat long lines in this package to 120 chars
We have many declarations and invocations..etc with long lines which are
very difficult to follow while doing code reading. This address the issues
in 'e2e/staticpvc.go' and 'e2e/snapshot.go' files to restrict the line length
to 120 chars.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-06-28 14:43:49 +00:00
Humble Chirammal
d6a4003981 e2e: reformat long lines in this package to 120 chars
We have many declarations and invocations..etc with long lines which are
very difficult to follow while doing code reading. This address the issues
in 'e2e/utils.go' files to restrict the line length to 120 chars.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-06-28 14:43:49 +00:00
Humble Chirammal
7aa53350ff e2e: reformat long lines in this package to 120 chars
We have many declarations and invocations..etc with long lines which are
very difficult to follow while doing code reading. This address the issues
in 'e2e/resize.go' files to restrict the line length to 120 chars.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-06-28 14:43:49 +00:00
Humble Chirammal
1849887b40 e2e: reformat long lines in this package to 120 chars
We have many declarations and invocations..etc with long lines which are
very difficult to follow while doing code reading. This address the issues
in 'e2e/pvc.go' file to restrict the line length to 120 chars.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-06-28 14:43:49 +00:00
Humble Chirammal
676b9f84c3 e2e: reformat long lines in this package to 120 chars
We have many declarations and invocations..etc with long lines which are
very difficult to follow while doing code reading. This address the issues
in 'e2e/pod.go' files to restrict the line length to 120 chars.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-06-28 14:43:49 +00:00
Humble Chirammal
abc993ce25 e2e: reformat long lines in this package to 120 chars
We have many declarations and invocations..etc with long lines which are
very difficult to follow while doing code reading. This address the issues
in 'e2e/deploy_vault.go' file to restrict the line length to 120 chars.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-06-28 14:43:49 +00:00
Humble Chirammal
971f8e320b e2e: reformat long lines in this package to 120 chars
We have many declarations and invocations..etc with long lines which are
very difficult to follow while doing code reading. This address the issues
in 'e2e/rbd_helper.go' files to restrict the line length to 120 chars.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-06-28 14:43:49 +00:00
Humble Chirammal
f07c5ab857 e2e: reformat long lines in this package to 120 chars
We have many declarations and invocations..etc with long lines which are
very difficult to follow while doing code reading. This address the issues
in 'e2e/rbd*.go' files to restrict the line length to 120 chars.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-06-28 14:43:49 +00:00
Humble Chirammal
49d11bf5ea e2e: reformat long lines in this package to 120 chars
We have many declarations and invocations..etc with long lines which are
very difficult to follow while doing code reading. This address the issues
in 'e2e/cephfs*.go' files to restrict the line length to 120 chars.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-06-28 14:43:49 +00:00
Rakshith R
82a204bfd3 e2e: add e2e for static PVC without imageFeature parameter
This commit adds e2e to make sure static PVC without imageFeatures
parameter fail with proper error event.

Signed-off-by: Rakshith R <rar@redhat.com>
2021-06-28 05:46:42 +00:00
Yati Padia
13667c013c cleanup: addresses paralleltest linter
The Go linter paralleltest checks that the t.Parallel
gets called for the test method and for the range of
test cases within the test.

Updates: #2025

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-06-25 11:55:12 +00:00
Humble Chirammal
0749315d7e e2e: make use of snapshot v1 api in utils.go
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-06-24 17:35:07 +00:00
Humble Chirammal
7823b360b8 e2e: make use of snapshot v1 api in cephfs_helper.go
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-06-24 17:35:07 +00:00
Humble Chirammal
f3d400730e e2e: make use of snapshot v1 api in cephfs.go
The import name of snapshot api package has also been changed
from `vs` to `snapapi` for consistency.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-06-24 17:35:07 +00:00
Humble Chirammal
5ba7790b26 e2e: make use of snapshot v1 api in rbd_helper.go
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-06-24 17:35:07 +00:00
Humble Chirammal
b7cd946b1e e2e: use snapshot v1 clientset and apis in snapshot.go
snapshot.go currently make use of snapshot v1beta1 clientset and api,
with this commit it has been rolled into v1 clientset and api.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-06-24 17:35:07 +00:00
Yug
a4548c3983 e2e: provide an option if tests run on helm deployment
add an e2eArg `helmTest` to specify if tests are running
on ceph-csi deployment via helm.
For testing in CI, Storageclass and secret deployment
is enabled on helm installation.

Signed-off-by: Yug <yuggupta27@gmail.com>
2021-06-24 11:51:56 +00:00
Humble Chirammal
8639015a9f cleanup: correct initResources() function name typo
This function was wrongly declared with name initResouces() in e2e
utils package and this patch address the typo in the name

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-06-23 12:15:40 +00:00
Humble Chirammal
9b6b83f73b e2e: remove bootstrapper BeforeEach and AfterEach
The e2e bootstrap does not make use of these or its declared
unwantedly in the same, removing it with this commit.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-06-22 14:15:14 +00:00
Yati Padia
847b996501 cleanup: Modifies Wrapcheck linter
Wrapcheck is a  simple Go linter to check that errors
from external packages are wrapped during return to
help identify the error source during debugging.
This commit addresses the wrapcheck error

Updates:#2025

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-06-22 08:47:55 +00:00
Niels de Vos
0fe0962dc1 e2e: validate thick-provisioned PVC-PVC cloning
Add a case to create a new PVC with VolumeContentSource set to a
thick-provisioned PVC. This should result in a new thick-provisioned PVC
once the cloning is done.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-06-18 06:25:56 +00:00
Madhu Rajanna
c6bc84d847 e2e: validate images in trash
when all the PVC and associated images are deleted,
the images should also get deleted from the trash.
This commit adds the validation check for the same.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-06-15 15:38:24 +00:00
Humble Chirammal
4302e5ee67 cleanup: correct createORDeleteCephfsResources() function name
Along with correcing the name of the function, other typos are also
addressed

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-06-11 08:04:07 +00:00
Humble Chirammal
a9fa108959 e2e: use proper variable name for rbd mount options
The variable naming for rbd mount options has been changed
to rbdMountOptions to be consistent with other variable naming schema

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-06-11 08:04:07 +00:00
Humble Chirammal
5635e99e37 e2e: correct gosec marker for credentials rule
The marker for hardcoded credentials check was set wrongly
and this patch address the same

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-06-11 08:04:07 +00:00
Humble Chirammal
2851c7d17a cleanup: correct createORdeleteRbdResources() function name
This patch address a typo in the mentioned function name

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-06-11 08:04:07 +00:00
Yug
33bf3fae6b e2e: add logErr function to log errors
logErr function logs all the ocured errors
with a message that is passed for occurence
of each error.

Co-authored-by: Niels de Vos <ndevos@redhat.com>
Co-authored-by: Madhu Rajanna <madhupr007@gmail.com>
Signed-off-by: Yug <yuggupta27@gmail.com>
2021-06-10 21:46:53 +00:00
Yug
8d038db047 e2e: add e2e to test clone in different pool
added a e2e to test clones in different
pool.

Co-authored-by: Madhu Rajanna <madhupr007@gmail.com>
Signed-off-by: Yug <yuggupta27@gmail.com>
2021-06-10 21:46:53 +00:00
Yug
1b2e9f556f e2e: add helper function for clone validation
added a helper function to test clone creation
in a different pool.

Co-authored-by: Madhu Rajanna <madhupr007@gmail.com>
Signed-off-by: Yug <yuggupta27@gmail.com>
2021-06-10 21:46:53 +00:00
Yug
91662b54b6 e2e: move yaml filepath declaration to global
declared yaml filepath in gobal for reusing.

Co-authored-by: Madhu Rajanna <madhupr007@gmail.com>
Signed-off-by: Yug <yuggupta27@gmail.com>
2021-06-10 21:46:53 +00:00
Yug
30092ff4e1 e2e: add an option to set rbd storageclass name
added an option to set storageclass name.

Co-authored-by: Madhu Rajanna <madhupr007@gmail.com>
Signed-off-by: Yug <yuggupta27@gmail.com>
2021-06-10 21:46:53 +00:00
Yug
76e9b4b02a e2e: add function to create rbd pool
added a new function to create rbd pool
in e2e.

Co-authored-by: Madhu Rajanna <madhupr007@gmail.com>
Signed-off-by: Yug <yuggupta27@gmail.com>
2021-06-10 21:46:53 +00:00
Yug
e302797217 e2e: update listRBDImages to list images in different pool
made pool as a argument of listRBDImages to support
listing of rbd images in different pools.

Co-authored-by: Madhu Rajanna <madhupr007@gmail.com>
Signed-off-by: Yug <yuggupta27@gmail.com>
2021-06-10 21:46:53 +00:00
Yati Padia
21a400839f cleanup: No use of variable validateEncryption
In the function validatePVCSnapshot(...), we don't need
validateEncryption variable as we are passing kms value
which can help us check the value of validateEncryption.
Hence, we can avoid using that.

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-06-08 13:00:11 +00:00
Yati Padia
0f44c6acb7 cleanup: address wasted assign issues
At places variable is reassigned without
being used.

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-06-03 09:51:14 +00:00
Rakshith R
81809500be e2e: wait for upgraded pods to be running during upgrade-tests
This commit calls `waitForDaemonSets` and `waitForDeploymentComplete`
after upgrading to wait for csi driver pods to be in running state
for both rbd and cephfs upgrade tests.

Signed-off-by: Rakshith R <rar@redhat.com>
2021-05-26 13:15:52 +00:00
Prasanna Kumar Kalever
6984da5096 build: ignore unparam linter false positive
Ignoring below warnings:

e2e/pod.go:207:60: `execCommandInContainer` - `cn` always receives
`"csi-rbdplugin"` (unparam)
func execCommandInContainer(f *framework.Framework, c, ns, cn string,
opt *metav1.ListOptions) (string, string, error) {
                                                           ^
e2e/pod.go:308:43: `deletePodWithLabel` - `skipNotFound` always receives
`false` (unparam)
func deletePodWithLabel(label, ns string, skipNotFound bool) error {

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-05-26 10:41:34 +00:00
Prasanna Kumar Kalever
85e1e0370a e2e: enable an old testcase as the ndb module is available
This testcase tests journaling/exclusive-lock image-features with
rbd-nbd mounter

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-05-26 10:41:34 +00:00
Prasanna Kumar Kalever
819267112e e2e: restart rbd-nbd process after nodeplugin reboot
Bringup the rbd-nbd map/attach process on the rbd node plugin and expect the
IO to continue uninterrupted.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-05-26 10:41:34 +00:00
Prasanna Kumar Kalever
7334c3b783 e2e: add ability to run command inside specified container
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-05-26 10:41:34 +00:00
Prasanna Kumar Kalever
695ec6dffe e2e: Test IO after nodeplugin reboot
This is a negative testcase to showcase as per current design
the IO will fail because of the missing mappings

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-05-26 10:41:34 +00:00
Prasanna Kumar Kalever
8bae8f8458 e2e: add a test case for rbd-nbd mounter
To validate the basic working of rbd-nbd

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-05-26 10:41:34 +00:00
Rakshith R
fa1414d98f cleanup: address ineffectual assignement linter issue
updates: #1586

Signed-off-by: Rakshith R <rar@redhat.com>
2021-05-26 07:04:32 +00:00
Rakshith R
b891e5585d cleanup: address ifshort linter issues
This commit addresses ifshort linter issues which
checks if short syntax for if-statements is possible.

updates: #1586

Signed-off-by: Rakshith R <rar@redhat.com>
2021-05-26 07:04:32 +00:00
Niels de Vos
2b9f6c3598 e2e: fetch volume metrics from Kubelet
Test if metrics are available at all. The actual values are a little
difficult to validate.

BlockMode volumes support metrics since Kubernetes 1.22.

See-also: kubernetes/kubernetes#97972
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-05-25 06:41:04 +00:00
Humble Chirammal
151b8c665f e2e: update upgrade-version to v3.3.1
making default version of upgrade test to v3.3.1

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-05-24 16:12:20 +00:00
Madhu Rajanna
fa36a46682 e2e: pvc mounting when snap and parent pvc is deleted
Added an E2E test to test below case

* Create PVC
* Create Snapshot from PVC
* Delete PVC
* Create Clone from Snapshot
* Delete Snapshot
* Mount clone to Application
* Delete Application and PVC Clone

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-05-07 07:57:37 +00:00
Madhu Rajanna
c3bae17fce rbd: delete encryption key from KMS
when a Snapshot is encrypted during a CreateSnapshot
operation, the encryption key gets created in the KMS
when we delete the Snapshot the key from the KMS
should also gets deleted.

When we create a volume from snapshot we are copying
required information but we missed to copy the
encryption information, This commit adds the missing
information to delete the encryption key.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-04-30 08:05:47 +00:00
Niels de Vos
35d58a7d5a e2e: only test a single encrypted clone/snapshot
The default number for cloning and snapshot/restore is 10 volumes. This
adds to the time the test suite runs. There is no need to validate 10
copies of the encrypted volume, a single copy is sufficient.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-04-14 03:59:28 +00:00
Niels de Vos
bb296c4f21 e2e: add verification for encrypted Snapshot/Restore operations
This moves validatePVCSnapshot() into its own function, so that it
follows the same format as validatePVCClone() does already.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-04-14 03:59:28 +00:00
Niels de Vos
3fde636685 e2e: add validation for cloning encrypted volumes
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-04-14 03:59:28 +00:00
Rakshith R
3f3489367c cleanup: correct linter name mnd to gomnd
Signed-off-by: Rakshith R <rar@redhat.com>
2021-04-09 07:24:47 +00:00
Yug
4f955e474d cleanup: correct misspelling
Correct snapshots spelling in rbd.go

Signed-off-by: Yug <yuggupta27@gmail.com>
2021-04-01 12:00:21 +00:00
Madhu Rajanna
fba6a2d0c3 deploy: add csidriver object for cephfs and rbd
csidriver object can be created on the kubernetes
for below reason.

If a CSI driver creates a CSIDriver object,
Kubernetes users can easily discover the CSI
Drivers installed on their cluster
(simply by issuing kubectl get CSIDriver)

Ref: https://kubernetes-csi.github.io/docs/csi-driver-object.html#what-is-the-csidriver-object

attachRequired is always required to be set to
true to avoid issue on RWO PVC.

more details about it at https://github.com/rook/rook/pull/4332

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-03-31 13:41:35 +00:00
Niels de Vos
296f751d08 e2e: verify (non)existence of keys for VaultTokensKMS
Key existence and removal is only checked for the VaultKMS provider. It
should also be done for the VaultTokensKMS provider.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-03-30 08:29:33 +00:00
Rakshith R
9e957520ae e2e: add test case for app deletion with unmounted cephFS volume
Checks app deletion when cephFS volume is already unmounted.
Creates app, pvc and binds them. Unmounts the volume through
umount cmd in cephFS plugin and checks app deletion.

Signed-off-by: Rakshith R <rar@redhat.com>
2021-03-25 15:15:07 +00:00
Rakshith R
e2dc8439e4 e2e: add execCommandInDaemonsetPod()
execCommandInDaemonsetPod() executes commands inside given
container of a daemonset pod on a particular node.

Signed-off-by: Rakshith R <rar@redhat.com>
2021-03-25 15:15:07 +00:00
Rakshith R
915437b314 e2e: add listPods()
listPods returns slice of pods matching given ListOptions and
namespace.

Signed-off-by: Rakshith R <rar@redhat.com>
2021-03-25 15:15:07 +00:00
Prasanna Kumar Kalever
e6a0049c0b e2e: add getDaemonSetLabelSelector() to utils
getDaemonSetLabelSelector returns labels of daemonset given name and
namespace dynamically, needed since labels are not same for helm and
non-helm deployments.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
Signed-off-by: Rakshith R <rar@redhat.com>
2021-03-25 15:15:07 +00:00
Rakshith R
26c07aef02 e2e: small fixes in var name and error msg
rookTolBoxPodLabel to rookToolBoxPodLabel
podPot to podOpt
fix error message for getting PV

Signed-off-by: Rakshith R <rar@redhat.com>
2021-03-25 15:15:07 +00:00
Humble Chirammal
82bc993b32 util: remove unwanted import string from module dependencies
There is no need for an extra import string when the go mod package
itself declared in the same.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-03-24 15:12:13 +00:00
Madhu Rajanna
d8f7b38d3d rbd: add exclusive-lock and journaling image features for rbd image
Current rbd plugin only supports the layering feature
for rbd image. Add exclusive-lock and journaling image
features for the rbd.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
Signed-off-by: woohhan <woohyung_han@tmax.co.kr>
2021-03-24 09:48:04 +00:00
Yati Padia
0d9548c815 Cephfs: Failed to delete snapshot
Failed to delete voluesnapshot when backend subvolume
(pvc) and ceph fs subvolume snapshot is deleted

Fixes#1647

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-03-17 10:28:08 +00:00
Yati Padia
cfb6abc067 e2e: update snapshot restore e2e to check data consistency
Currently, in rbd snapshot restore and volume clone E2E we
are not checking any data consistency after doing snapshot
restore or volume clone. Hence, this PR writes the data in
the PVC and checks the checksum of the file and verify it with
the snapshot or cloned PVC.

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-03-16 09:36:46 +00:00
Niels de Vos
c5f00a9e3d e2e: add test for the SecretsMetadataKMS provider
The new provider should be able to provision and mount an encrypted
volume.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-03-12 10:11:47 +00:00
Niels de Vos
abfe3ed9cb e2e: rework thick-provisioning test case
The stripe-size is the most efficient size to write to RBD images.
However, not all images are a multiple of stripe-size large. That means
thick-provisioning would not allocate the full image, and the process
might even fail.

This adds a 50 MB PVC to test the process, 100 MB is coincidentally a
multiple of the (default 4 MB) stripe-size, 50 MB is not.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-03-11 10:57:31 +00:00
Madhu Rajanna
a8fcf70cb8 e2e: make use of ceph users created in e2e
This commit adds support to create and delete
new ceph users to test various functionalities.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-03-03 03:44:41 +00:00
Niels de Vos
e261191f0b e2e: pass namespace once in deletePodWithLabel()
Currently framework.RunKubectl() adds `--namespace=...` 2x to the
kubectl command. Once is sufficient.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-03-02 14:44:41 +00:00
Madhu Rajanna
f4d5fdf114 e2e: add e2e for cephfs VolumeNamePrefix for PVC
when user provides an option for VolumeNamePrefix
create subvolume with the prefix which will be easy
for user to identify the subvolumes belongs to
the storageclass, Added an E2E testing to verify
that the subvolume contains the Prefix what is
provided in the storageclass.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-02-19 17:04:17 +00:00
Niels de Vos
2363d2a789 e2e: add validation for thick-provisioning
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-02-19 11:55:40 +00:00
Niels de Vos
8870575a48 e2e: compare resource.Quantity with Equals() instead of operator
When one Quantity is in GiB, and the other in Dec (bytes), the value
should be the same. However, by using ==, this is not the case. It is
needed to use Equals() for that.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-02-19 11:55:40 +00:00
Mudit Agarwal
4eb5bd6ba6 e2e: add e2e for snapshot retention fetaure.
Added e2e for snapshot retention feature.

Signed-off-by: Mudit Agarwal <muagarwa@redhat.com>
2021-02-19 10:03:55 +00:00
Niels de Vos
0d9cb96db1 e2e: log status of PVC when Status.Phase is empty
When deleting a PVC fails, the following messages are repeated until a
timeout is hit:

    cephfs-80811 in state &PersistentVolumeClaimStatus{Phase:,AccessModes:[],Capacity:ResourceList{},Conditions:[]PersistentVolumeClaimCondition{},} to be deleted (600 seconds elapsed)

Because the Phase is not set, the PVC seems to be in a strange state. In
case this happens, log all details from the PVC so that we can identify
additional conditions to check for completed deletion.

Updates: #1874
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-02-19 06:09:43 +00:00
Niels de Vos
47c6223b3a e2e: track deletion of PVC and PV more closely
When passing a pointer to a PVC and PV, the status of the deleted
objects is not logged correctly. The `PersistentVolumeClaim.Status` and
`PersistedVolume.Status` that is added to the logs contain the status of
the initially created object (reference to the PVC/PV). When the PVC/PV
is removed, there is no guarantee that the object is updated.

Logs show an empty (nullified) `PersistentVolumeClaim.Status`, which is
not helpful. Instead, use the returned PVC/PV from the `Get()` function
and use that for further logging. Even when the `.Status` struct from
the PVC/PV gets wiped, the returned object should have correct details.

Updates: #1874
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-02-17 12:24:31 +00:00
Niels de Vos
ff728eaf0d e2e: error out in case deploying Hashicorp Vault fails
Failures when deploying Hashicorp Vault are logged as informative. This
means that testing will continue, even if Vault will not be available.

Instead of logging the errors as INFO, use FAIL so that tests are not
run and the problems are identified early and obviously.

Updates: #1795
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-02-17 08:50:40 +00:00
Niels de Vos
5bcd5cb928 e2e: use secret with "encryptionPassphrase" for RBD tests
The e2e tests create a Secret for using with the RBD StorageClass.
However this Secret was not used, instead the Rook generated Secret was
linked in the StorageClass.

By using our own Secret from the examples, Rook should not touch it when
we make modifications. In addition, no modifications are needed for
encryption anymore, as these are included in the example.

Updates: #1795
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-02-12 12:02:14 +00:00
Madhu Rajanna
1aaccd0e5a e2e: add validation for subvolumePath
as we are adding the subvolumepath to the
PV spec. this commits adds an E2E testing
to verify it.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-02-11 11:31:22 +00:00
Prasanna Kumar Kalever
0d5efe9147 doc: add more example formats to run e2e tests
Added more example to run e2e and functional tests using `go test` and
`make` commands.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-01-20 12:29:59 +00:00
Seena Fallah
1f17d25407 e2e: add filesystem PVC resize test for radosNamespace
Signed-off-by: Seena Fallah <seenafallah@gmail.com>
2021-01-05 11:26:09 +00:00
Madhu Rajanna
1db3f03856 e2e: IsRetryableAPIError is not available in kubernetes 1.20.0
IsRetryableAPIError is not available in latest
kubernetes release ie 1.20.0 created a internal
function called isRetryableAPIError for the same.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-17 16:04:54 +00:00
Niels de Vos
4abe128bd8 e2e: log events from deployment namespace
When tests run and something goes wrong during deployment, not all
information is available. Logging the events from the namespace where
Ceph-CSI (and Vault) is deployed, might help with troubleshooting.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-17 11:40:57 +00:00
Niels de Vos
6e2d16ed1e e2e: add test for VaultTokensKMS support
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-14 14:45:09 +00:00
Niels de Vos
bccd5cef46 e2e: validate storing of csi.volume.owner attribute
The csi.volume.owner should get stored when the csi-provisioner sidecar
passes additional metadata. This option is now enabled by default, so
the owner (Kubernetes Namespace) of RBD images is expected to be
available.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-09 13:58:48 +00:00
Madhu Rajanna
7d3bff0654 e2e: enable topology based provisioning
as we are testing the topology based
provisioning in our E2E, we need to set
topology=true in E2E.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-08 16:09:25 +00:00
Madhu Rajanna
39b1f2b4d3 cleanup: fix mispell words
fixed mispell words in the repo.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-11-29 12:47:46 +05:30
Niels de Vos
eb63e796a8 e2e: deleted secrets from Vault are available, but empty
Once the Vault API removed a secret, the contents will have been wiped.
The key is still available, until it gets destroyed. This causes the e2e
test to detect an empty secret, and assume that it has not been deleted
yet.

By requesting the `data` field from the secret, an error is thrown in
case the secret has been wiped. This makes it possible for the e2e test
to detect that the secret has been removed and scheduled for destroying.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-29 04:03:59 +00:00
Madhu Rajanna
4fcd649fa7 e2e: add testing for controller
This commits adds an E2E testing
to verify the metadata created by controller,
We are not checking the generated omap data,
but we will be verify PVC resize and binding
pvc to application.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-11-28 18:50:00 +00:00
Madhu Rajanna
0dd152928d e2e: add option to set retainpolicy for rbd storageclass
added an option to set retain policy for
rbd storageclasses.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-11-28 18:50:00 +00:00
Madhu Rajanna
168526a906 e2e: use centos as image for normal user validation
Reduce the number of images that get pulled from Docker Hub. Use the
official CentOS container registry instead.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-11-19 16:00:33 +00:00
Madhu Rajanna
8d86bac9b7 e2e: set imagePullPolicy to ifNotPresent
If the imagePullPolicy is not set and the image
tag is empty or latest the image is always pulled.
This commit sets the policy to pull image if not
present.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-11-18 13:43:52 +00:00
Prasanna Kumar Kalever
fcaa332921 doc: improve e2e tests guide
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2020-11-11 13:18:05 +00:00
Niels de Vos
7caca1137f e2e: do not use Failf() to abort tests in a go-routine (util)
There are several go-routines where Failf() is called, which will cause
a Golang panic inside the Ginko test framework. Instead of aborting the
go-routine, capture the error and check for failures once all
go-routines have finished.

The CephFS tests have been updated already, this changs only affects the
validatePVCClone() utility function.

Updates: #1359
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-06 03:31:39 +00:00
Niels de Vos
45d64ab7d0 e2e: do not use Failf() to abort tests in a go-routine (rbd)
There are several go-routines where Failf() is called, which will cause
a Golang panic inside the Ginko test framework. Instead of aborting the
go-routine, capture the error and check for failures once all
go-routines have finished.

The CephFS tests have been updated already, this changs only affects the
RBD tests.

Updates: #1359
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-06 03:31:39 +00:00
Niels de Vos
96eafecad8 e2e: do not use Failf() to abort tests in a go-routine
There are several go-routines where Failf() is called, which will cause
a Golang panic inside the Ginko test framework. Instead of aborting the
go-routine, capture the error and check for failures once all
go-routines have finished.

Updates: #1359
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-05 11:26:20 +00:00
Niels de Vos
48108bc549 e2e: retry deploying on API-server timeouts
The upgrade-tests-cephfs fails relative regularly with the following
error during intial deployment:

    timeout waiting for deployment csi-cephfsplugin-provisioner with error error waiting for deployment "csi-cephfsplugin-provisioner" status to match expectation: etcdserver: request timed out

By detecting if the API-server returned a non-fatal error, the test does
not need to abort, but can wait for completion. PollImmediate() will
still return ErrWaitTimeout once the timeout elapsed.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-28 06:56:41 +00:00
Humble Chirammal
b4452f0787 e2e: validate the error return instead of object for consitency
If loadPVC() fails, it return error and we expect the PVC object
to be nil too. In many places we check on the error and exit.
However in few places we are looking at PVC object.
This commit make the condition check on `err` instead of `PVC`
object for consistency.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-10-21 13:56:12 +00:00
Humble Chirammal
dc4da25a30 e2e: correct typo in encryption string
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-10-21 13:56:12 +00:00
Niels de Vos
af922f267a e2e: use k8sVersionGreaterEquals() for rbd upgrade tests
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-19 16:00:39 +00:00
Niels de Vos
a2c077e0d5 e2e: use k8sVersionGreaterEquals() for cephfs upgrade tests
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-19 16:00:39 +00:00
Niels de Vos
cfddc9f887 e2e: use k8sVersionGreaterEquals() for cephfs tests
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-19 16:00:39 +00:00
Niels de Vos
d26030ca5f e2e: use k8sVersionGreaterEquals() for rbd tests
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-19 16:00:39 +00:00
Niels de Vos
c4289f01e8 e2e: add k8sVersionGreaterEquals() helper to check k8s versions
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-19 16:00:39 +00:00
Humble Chirammal
c0eca87268 e2e: make use of new helper function for filesystem mode PVC clone
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-10-15 15:37:14 +00:00
Humble Chirammal
25400a7334 e2e: Introduce e2e test for block mode PVC
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-10-15 15:37:14 +00:00
Humble Chirammal
5763e996af e2e: add validatePVCClone helper function
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-10-15 15:37:14 +00:00
Humble Chirammal
2f828cf20c e2e: update readme for new sidecar repository path
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-10-13 12:34:42 +00:00
Yug
c89afebd4a ci: Upgrade rbd upgrade tests
Add snapshot and pvc cloning test cases
to rbd upgrade testing.

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-09-30 18:07:15 +00:00
Yug
726dcf4da3 ci: Update cephfs upgrade tests
Add snapshot cloning and pvc cloning test cases for
cephfs upgrade tests.

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-09-30 18:07:15 +00:00
Yug
63b5a7ac96 ci: add function to validate checksum
calculateSha512sum() returns the md5sum of a file inside a pod.

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-09-30 18:07:15 +00:00
Nico Berlee
6a5f1380b0 deploy: add pod anti-affinity for provisioner deployments
The added anti-affinity rules prevent provisioner operators from scheduling on
the same nodes. The kubernetes scheduler will spread the pods across nodes to
improve availability during node failures.

Signed-off-by: Nico Berlee <nico.berlee@on2it.net>
2020-09-29 09:29:58 +00:00
Prasanna Kumar Kalever
27c8318a82 doc: update memory assigned to minikube vm
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2020-09-28 16:00:45 +00:00
Madhu Rajanna
dc38320fad e2e: add backend validation for rados namespaced images
added more backend validation for rados namespaced
images.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-09-25 15:35:22 +00:00
Niels de Vos
f41cd105c4 e2e: add simple ceph-fuse StorageClass tests
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-09-23 12:12:51 +00:00
Niels de Vos
85df59531b e2e: pass SC parameters to createCephfsStorageClass()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-09-23 12:12:51 +00:00
Madhu Rajanna
d1f175d9f3 rbd: add support for rbd map and unmap options
added support for providing map and unmap
options to rbd CLI when mapping rbd image
on the node.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-09-21 13:27:28 +00:00
Madhu Rajanna
be3a78afaf e2e: update cephfs ROX test case
updated cephfs ROX test case to create
ROX Clone PVC instead of creating an empty PVC

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-09-10 13:47:23 +00:00
Madhu Rajanna
b4693dcffe e2e: rework on E2E framework
rework of E2E framework for better code
organization and add more helpful logs for
debugging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-09-10 07:40:13 +00:00
Prasanna Kumar Kalever
b8bd233b15 doc: correct default kube version
fix kube version in e2e/README.md

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2020-08-28 03:59:58 +00:00
Madhu Rajanna
d5c19954c1 e2e: add backend image validation for rbd
validate backend rbd images count in each
E2E test cases. This helps a lot to catch
the issues in each test case.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-24 08:16:59 +00:00
Mudit Agarwal
46acd6b2cd doc: update e2e README for snapshot operation
Updated e2e README with the prerequisites to run snapshot/clone e2e tests.

Signed-off-by: Mudit Agarwal <muagarwa@redhat.com>
2020-08-12 16:27:49 +00:00
Mehdy Khoshnoody
204487383d e2e: Add e2e test cases for rbd rados namespace
These test cases are will be executed against a rados namespace.
- Create a PVC and bind it to an app.
- Resize block PVC and check device size.
- Create a PVC clone and bind it to an app.

Signed-off-by: Mehdy Khoshnoody <mehdy.khoshnoody@gmail.com>
2020-08-12 16:22:58 +05:30
Madhu Rajanna
b5320d9273 e2e: add backend validation for pvc restore and clone
added backend subvolume validation for cephfs
snapshot restore and pvc to pvc clone.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-11 16:21:29 +00:00
Madhu Rajanna
2307687dc2 e2e: add backend validation of cephfs pvc
added a backend validation to check subvolumes
count when we create and delete pvc.

fixes #1316

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-11 16:21:29 +00:00
Madhu Rajanna
141b89f514 e2e: add function to get subvolume list
added a helper function in e2e to list subvolumes
in a filesystem

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-11 16:21:29 +00:00
Humble Chirammal
ef1785ce4d e2e: remove volume populate test case and also change writeDataInPod()
As we are populating the volume in other two test cases for clone and
snapshot operation, we dont need a specific test case now.
WriteDataInPod() function is also changed to take the pod spec and write
some data to it

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-08 00:11:35 +00:00
Humble Chirammal
074feddaf3 e2e: add test cases for cephfs snapshot creation,deletion and restore
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-08 00:11:35 +00:00
Humble Chirammal
90762342a0 e2e: introduce createCephFSSnapshotClass creation
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-08 00:11:35 +00:00
Humble Chirammal
2d72550a39 e2e: introduce writeDataInPod() to write data to an attached PVC
writeDataInPod() write data to the attached PVC using `dd` command
It leave the pod and pvc state as it is.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-08 00:11:35 +00:00
Yug
02b4a7175c rbd: add upgrade testing
Upgrade testing will enable us to keep
in check the backward compatibility of
earlier releases.

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-07-26 03:41:53 +00:00
Yug
9b30969594 cephfs: add upgrade testing
Upgrade testing will enable us to keep
in check the backward compatibility of
earlier releases.

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-07-26 03:41:53 +00:00
Yug
9c0d5abb5a doc: Add README for upgrade-testing
Update README with upgrade testing parameters.

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-07-26 03:41:53 +00:00
Madhu Rajanna
5168ad7ddf e2e: create/delete snap and clone in parallel
In rbd E2E testing,we need to create snap and clone
as parallel operation.

This helps us to insure that functionality works when
we have parallel delete and create operations and also
it helps to catch bugs when we get parallel requests.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-21 13:25:19 +00:00
Yug
e73fe64a0d cleanup: address gosec warnings
gosec warns about security problems by scanning the
Go AST.

Issues Reported:
G101 (CWE-798): Potential hardcoded credentials (Confidence: LOW, Severity: HIGH)
G204 (CWE-78): Subprocess launched with variable (Confidence: HIGH, Severity: MEDIUM)
G304 (CWE-22): Potential file inclusion via variable (Confidence: HIGH, Severity: MEDIUM)

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-07-21 08:36:24 +00:00
Yug
628ae9e982 cleanup: use wrapped static errors instead of dynamic
In Go 1.13, the fmt.Errorf function supports a new %w verb.
When this verb is present, the error returned by fmt.Errorf
will have an Unwrap method returning the argument of %w,
which must be an error. In all other ways, %w is identical to %v.

Updates: #1227

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-07-21 08:36:24 +00:00
Yug
2bfe7670d2 cleanup: address govet warnings
We had "ns" as a parameter and then trying to
declare it also as a local variable, which is what
the complaint about "shadowing" refers to.

Issue reported:
shadow: declaration of "ns" shadows declaration at line 57 (govet)

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-07-21 08:36:24 +00:00
Yug
7f94a57908 cleanup: address godot warnings
Top level comments should end in a period

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-07-21 08:36:24 +00:00
Madhu Rajanna
9077c25c15 e2e: add e2e to test pvc-pvc cloning
Added an e2e testcase to test pvc-pvc
cloning.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-10 14:02:12 +00:00
Madhu Rajanna
b773e821c8 ci: use testCephFS flag to skip cephfs tests
This commit fixes the flag check to skip
cephfs E2E tests.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-09 13:50:21 +00:00
Madhu Rajanna
63b795d165 e2e: provide an option to skip rbd and cephfs e2e
Added two new parameters for e2e test to skip
rbd and cephfs tests. This will help us to
run more test in Travis CI.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-09 06:42:28 +00:00
Madhu Rajanna
6838d30aff e2e: reduce the pod cound and wait time
Reduced the number of pods created
in ROX E2E to save some time in E2E
and changed the waiting time from 2 to 1
min.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-06 10:08:31 +00:00
Niels de Vos
c9d5bc8a08 cleanup: use errors.As() in e2e/utils
See-also: https://github.com/golang/go/wiki/ErrorValueFAQ#how-should-i-change-my-error-handling-code-to-work-with-the-new-features
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-03 09:12:48 +00:00
Madhu Rajanna
47fb5f2299 e2e: update E2E testing for snapshot
with new implemntation when user creates a snapshot
in backend we are creating rbd image, we need to
validate the total images count in backend when
creating snapshots and clones.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-01 08:21:47 +00:00
Madhu Rajanna
21bd26ebc4 e2e: Add E2E for ROX PVC
Added E2E testing for creation
and mounting of ROX PVC, if the
PVC accessmode is ReadOnlyMany
the application pod should not get
write access to it.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-06-30 17:43:41 +00:00
Madhu Rajanna
37979da5b2 e2e: Add E2E for app readonly options
If the mount option is readonly in app
pod, the pod should not get the write
access to the mounted cephfs subvolume.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-06-30 17:43:41 +00:00
Yug Gupta
c9ad904331 cephfs: Add support for multiple subvolumegroups
With the current code base, the subvolumegroup will
be created once, and even for a different cluster,
subvolumegroup creation is not allowed again.

Added support multiple subvolumegroups creation by
validating one subvolumegroup creation per cluster.

Fixes: #1123

Signed-off-by: Yug Gupta <ygupta@redhat.com>
2020-06-26 09:19:24 +00:00
Madhu Rajanna
39561b9675 e2e: Add E2E to mount rbd PVC as ro in app
Added an E2E to mount rbd PVC as readonly
in application pod and try to create some
file in Readonly PVC,when we try to create
files on RO PVC, we should get error.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-06-22 06:15:40 +00:00
Madhu Rajanna
649aeb7aaf rbd: Add support for rbd ROX PVC mounting
if the PVC access mode is ReadOnlyMany
or single node readonly, mounting the rbd
device path to the staging path as readonly
to avoid the write operation.

If the PVC acccess mode is readonly, mapping
rbd images as readonly.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-06-22 06:15:40 +00:00
Humble Chirammal
dcbf71dd5d cleanup: avoid double import of kubernetes library in e2e utils
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-06-19 08:42:08 +00:00
Humble Chirammal
323cc0e3bb cleanup: avoid comparing errors directly
Go 1.13 contains support for error wrapping. To support wrapping,
fmt.Errorf now has a %w verb for creating wrapped errors, and three
new functions in the errors package ( errors.Unwrap, errors.Is and
errors.As) simplify unwrapping and inspecting wrapped errors.

With this change, If we currently compare errors using ==, we have to
use errors.Is instead. Example:

if err == io.ErrUnexpectedEOF

becomes

if errors.Is(err, io.ErrUnexpectedEOF)

https://tip.golang.org/doc/go1.13#error_wrapping

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-06-17 06:26:11 +00:00
Humble Chirammal
c1a269191c doc: make sure configmap object referred or documented correctly
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-06-10 11:33:10 +00:00
Madhu Rajanna
44da7ffb4e e2e: Add testcase of invalid mount option
Added a test case to cover unmap of rbd image
if the mounting fails. if we pass the invalid
mount option the expectation is that mounting
of rbd image to stagingpath fails. as the unmap
happens it should not block the rbd pvc deletion
saying rbd image is in-use.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-06-01 11:27:25 +00:00
Madhu Rajanna
78cdf9d4db e2e: Add option to set mountoptions in sc
Updated E2E code to pass mountoptions when
creating a storageclass.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-06-01 11:27:25 +00:00
Yug Gupta
753f6b7fec cleanup: fix golint warnings in util, e2e
util: golint warns about exported methods to have a
comment or to unexport them.

e2e: golint warns about package comment to be of the form
"Package e2e ..."

Reported-by: https://goreportcard.com/report/github.com/ceph/ceph-csi

Updates: #975

Signed-off-by: Yug Gupta <ygupta@redhat.com>
2020-05-20 13:18:41 +00:00
Madhu Rajanna
b0270ced41 e2e: reword sentences in e2e
as kube is the shortform for kubernetes.
its expected to mention full form kubernetes
in the e2e tests.

Updated few wordings in the e2e.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-05-20 06:37:36 +00:00
Madhu Rajanna
7a5e064f50 e2e: Add a new function to run commands in toolboxpod
Updated code to use new function execCommandInToolBoxPod
to run commands in toolboxpod.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-05-20 06:37:36 +00:00
Madhu Rajanna
0f6c6c15a8 e2e: remove explict setting of kind and apiversion
we dont need to explictly set the kind
and apiversion in the snapshot class object
as it is already set in snapshotclass.yaml file

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-05-20 06:37:36 +00:00
Mudit Agarwal
fb1e689648 test: add e2e test for cephFS static PV
With the change in https://github.com/ceph/ceph-csi/issues/382, support for static PV for CephFS was added.
This change is to add e2e test case for the same to ensure that we do not regress the functionality.

Issue: https://github.com/ceph/ceph-csi/issues/669

Signed-off-by: Mudit Agarwal <muagarwa@redhat.com>
2020-05-18 08:30:50 +00:00
Madhu Rajanna
f0b3cee94a e2e: define replicapool as constant
the static check is failing as the replicapool
is used in 3 or more places, we need to define
a variable and use it.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-05-15 08:19:32 +00:00
Madhu Rajanna
76a55350aa e2e: Pass context.TODO() to kubernetes snapshot client
in external-snapshotter the client has changed we
need to pass the context to snapshot api calls.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-05-15 08:19:32 +00:00
Madhu Rajanna
1f13692000 ci: skip snapshot E2E if kube<1.17+
snapshot beta CRD wont work if the
kubernetes version is less than 1.17.0
as the snapshot CRD wont be installed
we cannot test the snapshot,so disabling
it if the kube version is less than 1.17

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-05-15 08:19:32 +00:00
Madhu Rajanna
c5bf95b625 e2e: Enable E2E for rbd snapshot
Updated E2E to use beta client and beta
Snapshot resources.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-05-15 08:19:32 +00:00
Yug Gupta
2cdf5c3b9f util: make util.ClusterInfo usable outside util package
functions like getClusterInfo() returns struct
instead of a set of strings.

Fix: #998

Signed-off-by: Yug Gupta <ygupta@redhat.com>
2020-05-14 19:43:21 +00:00
Mathias Merscher
0991cdf498 make CephFS SubvolumeGroup configurable
The name of the CephFS SubvolumeGroup for the CSI volumes was hardcoded to "csi". To make permission management in multi tenancy environments easier, this commit makes it possible to configure the CSI SubvolumeGroup.

related to #798 and #931
2020-05-04 05:50:06 +00:00
Niels de Vos
7d642b791c e2e: fix IneffAssign warnings in checkPVSelectorValuesForPVC()
IneffAssign warns about the two following statements:

  Line 1342: warning: ineffectual assignment to rFound (ineffassign)
  Line 1350: warning: ineffectual assignment to zFound (ineffassign)

rFound and zFound should be set before entering the loop, otherwise the
initial value will overwrite the updated value on each iteration.

Reported-by: https://goreportcard.com/report/github.com/ceph/ceph-csi
Updates: #975
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-04-24 08:35:36 +00:00
ShyamsundarR
1a8f8e3c24 Add support for erasure coded pools
This commit adds support to mention dataPool parameter for the
topology constrained pools in the StorageClass, that can be
leveraged to mention erasure coded pool names to use for RBD
data instead of the replica pools.

Signed-off-by: ShyamsundarR <srangana@redhat.com>
2020-04-14 14:14:29 +00:00
ShyamsundarR
e73921f268 Add e2e tests for topology based provisioning
- This commit adds tests only for RBD, as CephFS still needs
an enhancement in CephFS subvolume commands to effectively use
topology based provisioning

Signed-off-by: ShyamsundarR <srangana@redhat.com>
2020-04-14 14:14:29 +00:00
ShyamsundarR
5c4abf8347 Add topology support to ceph-csi
Signed-off-by: ShyamsundarR <srangana@redhat.com>
2020-04-14 14:14:29 +00:00
Humble Chirammal
4c96ad3c85 Changes to e2e to accomodate client-go changes and RunKubectlInput
With client-go v1.18.0 there is a change where Signatures on methods
in generated clientsets, dynamic, metadata, and scale clients have been
modified to accept context.Context as a first argument.
Signatures of Create, Update, and Patch methods have been updated to accept
CreateOptions, UpdateOptions and PatchOptions respectively.
Signatures of Delete and DeleteCollection methods now accept DeleteOptions
by value instead of by reference

The framework.RunkubectlInput now accepts namespace as the first parameter
which is also accommodated with this PR.

Signed-off-by: Humble Chirammal hchiramm@redhat.com
2020-04-14 10:50:12 +00:00
Madhu Rajanna
22f3dc3556 E2E: Add Ability to test with different user #919
Updated E2E to use a normal user other than admin user,
who is having access to create and mount cephfs PVC and
create and map rbd PVC

we will use the user created by rook who is having above
access.

Signed-off-by: Madhu Rajanna madhupr007@gmail.com

This PR adds a test case for #904

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-04-09 15:10:55 +00:00
Madhu Rajanna
a1f5425b88 Update cephcsi secret created by rook
as we need to include the encryption
secret key inside the secret created by the
rook, This PR will add the key and value required
for the encryption inside the secrets.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-04-09 15:10:55 +00:00
Madhu Rajanna
d09ffbd6de helm: add helm charts E2E
This PR adds the support for helm
installation, and cephcsi helm charts
deployment and teardown and also runs E2E
on for helm charts.

Add socat to provide port forwadring access for helm

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-04-06 11:01:25 +00:00
Madhu Rajanna
b2dfcae802 Update configmap if already present
in e2e if the configmap is ready present,
we need to update it to make life simpler
for helm chart e2e.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-04-06 11:01:25 +00:00
Madhu Rajanna
e788328750 Refractor E2E to reduce code duplication
Updated E2E to reduce code duplication
and create resouces in different namespaces.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-03-20 10:16:23 +00:00
Madhu Rajanna
4690dc6ea7 Fix CI warning related to gosec
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-03-20 10:16:23 +00:00
Madhu Rajanna
24405dc178 Fix undeclared golangci-lint warnings
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-03-20 10:16:23 +00:00
Madhu Rajanna
c45c426215 Add cephcsi namespace and rook namespace flag
Added namespace flag to cephcsi to deploy cephcsi
resouces in different namespace.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-03-20 10:16:23 +00:00