Commit Graph

288 Commits

Author SHA1 Message Date
Rakshith R
a781ebb844 e2e: modify validatePVCSnapshot() to use restoreSCName & restoreKMS
Signed-off-by: Rakshith R <rar@redhat.com>
(cherry picked from commit f63ed2ca5a)
2021-10-05 11:33:26 +00:00
Rakshith R
342867a197 e2e: create reusable variable vaultUserSecretPath = "user-secret.yaml"
Signed-off-by: Rakshith R <rar@redhat.com>
(cherry picked from commit 1d49b6a288)
2021-08-11 09:50:10 +00:00
Rakshith R
0593071dac e2e: add modification to test encrypted PVC with rbd controller
Signed-off-by: Rakshith R <rar@redhat.com>
(cherry picked from commit 2f995eada2)
2021-08-11 09:50:10 +00:00
Rakshith R
f97c3f901d e2e: use retryKubectlFile() for creating & deleting secrets
Signed-off-by: Rakshith R <rar@redhat.com>
(cherry picked from commit 8ca7a35820)
2021-08-11 09:50:10 +00:00
Rakshith R
33899663e1 e2e: add prefixname to rbd controller test
Signed-off-by: Rakshith R <rar@redhat.com>
(cherry picked from commit 0744ad502b)
2021-08-11 09:50:10 +00:00
Madhu Rajanna
fbc1e5f3d5 e2e: retry running kubectl on known errors
By using retryKubectl helper function,
a retry will be done, and the known error
messages will be skipped.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit 2c66dfc3e4)
2021-08-11 07:03:05 +00:00
Madhu Rajanna
f7e150b84f e2e: pass variadic argument to kubectl helper function
this provides caller ability to pass the arguments
like ignore-not-found=true etc when executing
the kubectl commands.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit 2071c535fa)
2021-08-11 07:03:05 +00:00
Madhu Rajanna
64937f1f68 e2e: add retryKubectlArgs helper for kubectl retry
added helper function retryKubectlArgs to perform
action if its a known error.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit 9f0af30735)
2021-08-11 07:03:05 +00:00
Madhu Rajanna
9e84583063 e2e: add isAlreadyExistsCLIError to check known error
added isAlreadyExistsCLIError to check for known error.
if error is already exists we are considering it
as a success.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit dd9fabf747)
2021-08-11 07:03:05 +00:00
Rakshith R
05622b87c0 e2e: log imageList in validateRBDImageCount for better debugging
Signed-off-by: Rakshith R <rar@redhat.com>
(cherry picked from commit 7f6b73e71f)
2021-08-09 12:10:42 +00:00
Rakshith R
f5e73009e1 e2e: add test cases for pvc-pvcClone chain with depth 2
Signed-off-by: Rakshith R <rar@redhat.com>
(cherry picked from commit 9d57717222)
2021-08-09 12:10:42 +00:00
Rakshith R
dc046d0204 e2e: add test cases for snapshot-restore chain with depth 2
Signed-off-by: Rakshith R <rar@redhat.com>
(cherry picked from commit 9321b4bce4)
2021-08-09 12:10:42 +00:00
Niels de Vos
3bbcda6174 e2e: use official CentOS container location
registry.centos.org is not officially maintained by the CentOS
infrastructure team. The container images on quay.io are the official
once and we should use those instead.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
(cherry picked from commit e0ac70f8fb)
2021-08-04 10:50:30 +00:00
Madhu Rajanna
080b251850 e2e: validate images in trash for rados namespace
added validation check to verify stale images in trash
for the rados namespace testing.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-07-28 03:48:33 +00:00
Niels de Vos
ec6703ed58 rbd: rename encryption metadata keys to enable mirroring
RBD image metadata keys that start with '.rbd' are expected to be
internal to RBD itself and are not mirrored to remote sites. Renaming
the keys (dropping the '.' prefix) and using the new MigrateMetadata()
function now makes the keys available on remote sites too.

Closes: #2219
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-07-26 11:49:56 +00:00
Niels de Vos
e75d308b9c e2e: isRetryableAPIError() should match any etcdserver timeout
framework.RunKubectl() returns an error that does not end with
"etcdserver: request timed out", but contains the text somewhere in the
middle:

    error running /usr/bin/kubectl --server=https://192.168.39.57:8443 --kubeconfig=/root/.kube/config --namespace=cephcsi-e2e-a44ec4b4 create -f -:
    Command stdout:

    stderr:
    Error from server: error when creating "STDIN": etcdserver: request timed out

    error:
    exit status 1

isRetryableAPIError() should  return `true` for this case as well, so
instead of using HasSuffix(), we'll use Contains().

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-07-23 12:20:16 +00:00
Prasanna Kumar Kalever
75dda7ac0d e2e: add test for expansion of encrypted volumes
Also adds a test case to validate the default encryption type

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-07-23 10:00:23 +00:00
Yati Padia
1ae2afe208 cleanup: modifies the error caused due to merged PRs
This commit modifies the error of godot, cyclop,
paralleltest linter caused due to merged PRs.

Updates: #1586

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-07-22 18:15:48 +00:00
Yati Padia
172b66f73f cleanup: resolves cyclop linter issue
this commit adds `// nolint:cyclop` for the
fucntions whose complexity is above 20

Updates: #1586

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-07-22 18:15:48 +00:00
Rakshith R
43f753760b cleanup: resolve nlreturn linter issues
nlreturn linter requires a new line before return
and branch statements except when the return is alone
inside a statement group (such as an if statement) to
increase code clarity. This commit addresses such issues.

Updates: #1586

Signed-off-by: Rakshith R <rar@redhat.com>
2021-07-22 06:05:01 +00:00
Niels de Vos
5c016b4b94 e2e: retry on "connect: connection refused" errors
Sometimes there are failures in the e2e suite when connecting to the
etcdserver fails. The following error was caught:

    INFO: Error getting pvc "rbd-pvc" in namespace "rbd-1318": Get "https://192.168.39.222:8443/api/v1/namespaces/rbd-1318/persistentvolumeclaims/rbd-pvc": dial tcp 192.168.39.222:8443: connect: connection refused
    FAIL: failed to create PVC with error failed to get pvc: Get "https://192.168.39.222:8443/api/v1/namespaces/rbd-1318/persistentvolumeclaims/rbd-pvc": dial tcp 192.168.39.222:8443: connect: connection refused

If etcdserver was only briefly unavailable, one or more retries might be
sufficient to have the test pass.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-07-21 13:08:41 +00:00
Yati Padia
7f5df7c940 cleanup: resolves gofumpt issues in e2e
This commit resolves gofumpt issues in
e2e folder.

Updates: #1586

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-07-20 15:37:58 +00:00
Niels de Vos
841a53bc3d e2e: retry kubectl commands in case deploying Vault fails
Sometimes it happens that the deployment of Hashicorp Vault fails.
Deployment is one of the 1st steps that are done when starting the e2e
suite, and the Kubernetes cluster may still be a little overloaded while
it is settling down. It should be possible to retry and succeed after a
while.

Fixes: #2288
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-07-19 16:12:18 +00:00
Niels de Vos
d5ea89e603 e2e: add retryKubectlInput() for retrying kubectl calls
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-07-19 16:12:18 +00:00
Yati Padia
3469dfc753 cleanup: resolve errorlint issues
This commit resolves errorlint issues
which checks for the code that will cause
problems with the error wrapping scheme.

Updates: #1586

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-07-19 13:31:29 +00:00
Yati Padia
bfda5fa57f cleanup: resolve revive linter issue
revive linter checks for var-declaration
format.
For example:
"e2e/rbd_helper.go:441:36: var-declaration:
should drop = nil from declaration of
var noPVCValidation; it is the zero value (revive)
var noPVCValidation validateFunc = nil"

Updates: #1586

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-07-19 08:39:32 +00:00
Prasanna Kumar Kalever
10fc639d68 ci: fix nolintlint warnings
warnings from golangci-lint:

e2e/pod.go:207:122: directive `//nolint:unparam,lll // cn can be used
with different inputs later` is unused for linter unparam (nolintlint)
func execCommandInContainer(f *framework.Framework, c, ns, cn string,
opt *metav1.ListOptions) (string, string, error) { //nolint:unparam,lll
// cn can be used with different inputs later

e2e/pod.go:307:70: directive `//nolint:unparam // skipNotFound can be
used with different inputs later` is unused for linter unparam (nolintlint)
func deletePodWithLabel(label, ns string, skipNotFound bool) error {
//nolint:unparam // skipNotFound can be used with different inputs later

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-07-16 16:30:58 +00:00
Prasanna Kumar Kalever
fd3bf1750b e2e: fix the testcases for rbd-nbd
Now that the healer functionaity for mounter processes is available,
lets start, using it.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-07-16 16:30:58 +00:00
Niels de Vos
4d4a2a7814 e2e: prevent re-using empty pvc object
When an error occurs, the pvc object is overwritten in the
PollImmediate() loop. Re-using the pvc.Namespace results in error
messages like

    Error getting pvc in namespace: '': an empty namespace may not be set when a resource name is provided

and prevents the retry by PollImmediate() to never succeed. Storing the
namespace in a local variable prevents this from happening.

Reported-by: Rakshith R <rar@redhat.com>
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-07-14 10:18:51 +00:00
Niels de Vos
f7ae33c67c e2e: only call error check functions when err != nil
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-07-14 10:18:51 +00:00
Niels de Vos
075a4087d7 e2e: mark "etcdserver: request timed out" errors as retryable
There are regular CI failures where etcdserver times out. These errors
seem not to get caught by any of the existing error comparing. Matching
the error by string should prevent temporary etcdserver issues now too.

Updates: #2218
Closes: #1969
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-07-14 10:18:51 +00:00
Niels de Vos
e3c7dea7d6 e2e: add test for Vault with ServiceAccount per Tenant
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-07-13 17:16:35 +00:00
Yati Padia
4a649fe17f cleanup: resolve godot linter
This commit resolves godot linter issue
which says "Comment should end in a period (godot)".

Updates: #1586

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-07-13 06:50:03 +00:00
Niels de Vos
855f9080d1 e2e: refactor KMS configuration and usage
This adds a new `kmsConfig` interface that can be used to validate
different KMS services and setting. It makes checking for the available
support easier, and fetching the passphrase simpler.

The basicKMS mirrors the current validation of the KMS implementations
that use secrets and metadata. vaultKMS can be used to validate the
passphrase stored in a Vault service.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-07-12 14:57:39 +00:00
Yati Padia
84c1fe52c7 cleanup: resolve exhaustive linter
This commit resolves exhaustive linter
error.

Updates: #2240

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-07-12 04:47:08 +00:00
Rakshith R
b27d6319ca e2e: add e2e for user secret based metadata encryption
This commit adds e2e for user secret based metadata encryption,
adds user-secret.yaml and makes required changes in kms-connection-details,
kms-config yamls.

Signed-off-by: Rakshith R <rar@redhat.com>
2021-07-08 17:06:02 +00:00
Yati Padia
ffab37f44f cleanup: resolves gocritic linter issues
This commit resolves gocritic linter errors.

Updates: #2250

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-07-08 05:19:26 +00:00
Madhu Rajanna
f471a206c5 e2e: update e2e to validate thick metadata
updated e2e tests to validate the thick
metadata set on the rbd image for thick PVC.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-07-07 08:31:10 +00:00
Yug
e14d649547 e2e: add cephfs test for sc, secret via helm
add cephfs test for validating storageclass and
secret deployment via helm.

Signed-off-by: Yug <yuggupta27@gmail.com>
2021-07-06 10:55:41 +00:00
Yug
7d28896e36 e2e: add rbd test for testing sc, secret via helm
add rbd test for validating storageclass and
secret deployment via helm.

Signed-off-by: Yug <yuggupta27@gmail.com>
2021-07-06 10:55:41 +00:00
Rakshith R
fb365005da e2e: update e2epv.WaitOnPVandPVC() to match new parameters
Added &framework.TimeoutContext{ClaimBound: timeout, PVBound: timeout}
to parameters of function call in updated package.

Signed-off-by: Rakshith R <rar@redhat.com>
2021-07-01 03:35:23 +00:00
Madhu Rajanna
3e9172ab31 e2e: check isRetryableAPIError during polling
check isRetryableAPIError error when the
function is polling for the resouce.

updates: #1969

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-06-30 10:00:44 +00:00
Yati Padia
3c773b24e5 e2e: validate encryption keys in KMS
this commit is to validate if the encrypted
keys are created and deleted properly while
pvc-pvc clone images

Updates: #2022

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-06-30 08:21:01 +00:00
Humble Chirammal
36f4c0cabb e2e: remove passing waitgroups for go routines
We are unwantedly passing around waitgroups which is not required.
This address the same inside e2e/cephfs.go and remove waitgroup arg

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-06-29 15:07:56 +00:00
Humble Chirammal
4657ac7f00 e2e: remove passing waitgroups for go routines
We are unwantedly passing around waitgroups which is not required.
This address the same inside e2e/utils.go and remove waitgroup arg

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-06-29 15:07:56 +00:00
Humble Chirammal
053f7ee213 e2e: remove passing waitgroups reference for go routines
We are unwantedly passing around waitgroups which is not required.
This address the same inside e2e/rbd_helper.go and remove the waitgroup
arg

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-06-29 15:07:56 +00:00
Yati Padia
7f7a41d23a cleanup: use %w to format errors
Use %w instead of %v to format errors.

Updates: #1586

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-06-29 13:05:31 +00:00
Yati Padia
c851b69160 cleanup: error is not nil but returns nil
In few places the error is not nil still it
returns nil.

Updates: #1586

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-06-28 17:54:38 +00:00
Humble Chirammal
c2cda4909b e2e: reformat long lines in this package to 120 chars
We have many declarations and invocations..etc with long lines which are
very difficult to follow while doing code reading. This address the issues
in 'e2e/upgrade_*.go' files to restrict the line length to 120 chars.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-06-28 14:43:49 +00:00
Humble Chirammal
41ecaadc82 e2e: reformat long lines in this package to 120 chars
We have many declarations and invocations..etc with long lines which are
very difficult to follow while doing code reading. This address the issues
in 'e2e/staticpvc.go' and 'e2e/snapshot.go' files to restrict the line length
to 120 chars.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-06-28 14:43:49 +00:00