Commit Graph

174 Commits

Author SHA1 Message Date
Niels de Vos
abfe3ed9cb e2e: rework thick-provisioning test case
The stripe-size is the most efficient size to write to RBD images.
However, not all images are a multiple of stripe-size large. That means
thick-provisioning would not allocate the full image, and the process
might even fail.

This adds a 50 MB PVC to test the process, 100 MB is coincidentally a
multiple of the (default 4 MB) stripe-size, 50 MB is not.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-03-11 10:57:31 +00:00
Madhu Rajanna
a8fcf70cb8 e2e: make use of ceph users created in e2e
This commit adds support to create and delete
new ceph users to test various functionalities.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-03-03 03:44:41 +00:00
Niels de Vos
e261191f0b e2e: pass namespace once in deletePodWithLabel()
Currently framework.RunKubectl() adds `--namespace=...` 2x to the
kubectl command. Once is sufficient.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-03-02 14:44:41 +00:00
Madhu Rajanna
f4d5fdf114 e2e: add e2e for cephfs VolumeNamePrefix for PVC
when user provides an option for VolumeNamePrefix
create subvolume with the prefix which will be easy
for user to identify the subvolumes belongs to
the storageclass, Added an E2E testing to verify
that the subvolume contains the Prefix what is
provided in the storageclass.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-02-19 17:04:17 +00:00
Niels de Vos
2363d2a789 e2e: add validation for thick-provisioning
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-02-19 11:55:40 +00:00
Niels de Vos
8870575a48 e2e: compare resource.Quantity with Equals() instead of operator
When one Quantity is in GiB, and the other in Dec (bytes), the value
should be the same. However, by using ==, this is not the case. It is
needed to use Equals() for that.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-02-19 11:55:40 +00:00
Mudit Agarwal
4eb5bd6ba6 e2e: add e2e for snapshot retention fetaure.
Added e2e for snapshot retention feature.

Signed-off-by: Mudit Agarwal <muagarwa@redhat.com>
2021-02-19 10:03:55 +00:00
Niels de Vos
0d9cb96db1 e2e: log status of PVC when Status.Phase is empty
When deleting a PVC fails, the following messages are repeated until a
timeout is hit:

    cephfs-80811 in state &PersistentVolumeClaimStatus{Phase:,AccessModes:[],Capacity:ResourceList{},Conditions:[]PersistentVolumeClaimCondition{},} to be deleted (600 seconds elapsed)

Because the Phase is not set, the PVC seems to be in a strange state. In
case this happens, log all details from the PVC so that we can identify
additional conditions to check for completed deletion.

Updates: #1874
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-02-19 06:09:43 +00:00
Niels de Vos
47c6223b3a e2e: track deletion of PVC and PV more closely
When passing a pointer to a PVC and PV, the status of the deleted
objects is not logged correctly. The `PersistentVolumeClaim.Status` and
`PersistedVolume.Status` that is added to the logs contain the status of
the initially created object (reference to the PVC/PV). When the PVC/PV
is removed, there is no guarantee that the object is updated.

Logs show an empty (nullified) `PersistentVolumeClaim.Status`, which is
not helpful. Instead, use the returned PVC/PV from the `Get()` function
and use that for further logging. Even when the `.Status` struct from
the PVC/PV gets wiped, the returned object should have correct details.

Updates: #1874
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-02-17 12:24:31 +00:00
Niels de Vos
ff728eaf0d e2e: error out in case deploying Hashicorp Vault fails
Failures when deploying Hashicorp Vault are logged as informative. This
means that testing will continue, even if Vault will not be available.

Instead of logging the errors as INFO, use FAIL so that tests are not
run and the problems are identified early and obviously.

Updates: #1795
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-02-17 08:50:40 +00:00
Niels de Vos
5bcd5cb928 e2e: use secret with "encryptionPassphrase" for RBD tests
The e2e tests create a Secret for using with the RBD StorageClass.
However this Secret was not used, instead the Rook generated Secret was
linked in the StorageClass.

By using our own Secret from the examples, Rook should not touch it when
we make modifications. In addition, no modifications are needed for
encryption anymore, as these are included in the example.

Updates: #1795
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-02-12 12:02:14 +00:00
Madhu Rajanna
1aaccd0e5a e2e: add validation for subvolumePath
as we are adding the subvolumepath to the
PV spec. this commits adds an E2E testing
to verify it.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-02-11 11:31:22 +00:00
Prasanna Kumar Kalever
0d5efe9147 doc: add more example formats to run e2e tests
Added more example to run e2e and functional tests using `go test` and
`make` commands.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-01-20 12:29:59 +00:00
Seena Fallah
1f17d25407 e2e: add filesystem PVC resize test for radosNamespace
Signed-off-by: Seena Fallah <seenafallah@gmail.com>
2021-01-05 11:26:09 +00:00
Madhu Rajanna
1db3f03856 e2e: IsRetryableAPIError is not available in kubernetes 1.20.0
IsRetryableAPIError is not available in latest
kubernetes release ie 1.20.0 created a internal
function called isRetryableAPIError for the same.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-17 16:04:54 +00:00
Niels de Vos
4abe128bd8 e2e: log events from deployment namespace
When tests run and something goes wrong during deployment, not all
information is available. Logging the events from the namespace where
Ceph-CSI (and Vault) is deployed, might help with troubleshooting.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-17 11:40:57 +00:00
Niels de Vos
6e2d16ed1e e2e: add test for VaultTokensKMS support
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-14 14:45:09 +00:00
Niels de Vos
bccd5cef46 e2e: validate storing of csi.volume.owner attribute
The csi.volume.owner should get stored when the csi-provisioner sidecar
passes additional metadata. This option is now enabled by default, so
the owner (Kubernetes Namespace) of RBD images is expected to be
available.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-09 13:58:48 +00:00
Madhu Rajanna
7d3bff0654 e2e: enable topology based provisioning
as we are testing the topology based
provisioning in our E2E, we need to set
topology=true in E2E.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-08 16:09:25 +00:00
Madhu Rajanna
39b1f2b4d3 cleanup: fix mispell words
fixed mispell words in the repo.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-11-29 12:47:46 +05:30
Niels de Vos
eb63e796a8 e2e: deleted secrets from Vault are available, but empty
Once the Vault API removed a secret, the contents will have been wiped.
The key is still available, until it gets destroyed. This causes the e2e
test to detect an empty secret, and assume that it has not been deleted
yet.

By requesting the `data` field from the secret, an error is thrown in
case the secret has been wiped. This makes it possible for the e2e test
to detect that the secret has been removed and scheduled for destroying.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-29 04:03:59 +00:00
Madhu Rajanna
4fcd649fa7 e2e: add testing for controller
This commits adds an E2E testing
to verify the metadata created by controller,
We are not checking the generated omap data,
but we will be verify PVC resize and binding
pvc to application.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-11-28 18:50:00 +00:00
Madhu Rajanna
0dd152928d e2e: add option to set retainpolicy for rbd storageclass
added an option to set retain policy for
rbd storageclasses.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-11-28 18:50:00 +00:00
Madhu Rajanna
168526a906 e2e: use centos as image for normal user validation
Reduce the number of images that get pulled from Docker Hub. Use the
official CentOS container registry instead.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-11-19 16:00:33 +00:00
Madhu Rajanna
8d86bac9b7 e2e: set imagePullPolicy to ifNotPresent
If the imagePullPolicy is not set and the image
tag is empty or latest the image is always pulled.
This commit sets the policy to pull image if not
present.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-11-18 13:43:52 +00:00
Prasanna Kumar Kalever
fcaa332921 doc: improve e2e tests guide
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2020-11-11 13:18:05 +00:00
Niels de Vos
7caca1137f e2e: do not use Failf() to abort tests in a go-routine (util)
There are several go-routines where Failf() is called, which will cause
a Golang panic inside the Ginko test framework. Instead of aborting the
go-routine, capture the error and check for failures once all
go-routines have finished.

The CephFS tests have been updated already, this changs only affects the
validatePVCClone() utility function.

Updates: #1359
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-06 03:31:39 +00:00
Niels de Vos
45d64ab7d0 e2e: do not use Failf() to abort tests in a go-routine (rbd)
There are several go-routines where Failf() is called, which will cause
a Golang panic inside the Ginko test framework. Instead of aborting the
go-routine, capture the error and check for failures once all
go-routines have finished.

The CephFS tests have been updated already, this changs only affects the
RBD tests.

Updates: #1359
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-06 03:31:39 +00:00
Niels de Vos
96eafecad8 e2e: do not use Failf() to abort tests in a go-routine
There are several go-routines where Failf() is called, which will cause
a Golang panic inside the Ginko test framework. Instead of aborting the
go-routine, capture the error and check for failures once all
go-routines have finished.

Updates: #1359
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-05 11:26:20 +00:00
Niels de Vos
48108bc549 e2e: retry deploying on API-server timeouts
The upgrade-tests-cephfs fails relative regularly with the following
error during intial deployment:

    timeout waiting for deployment csi-cephfsplugin-provisioner with error error waiting for deployment "csi-cephfsplugin-provisioner" status to match expectation: etcdserver: request timed out

By detecting if the API-server returned a non-fatal error, the test does
not need to abort, but can wait for completion. PollImmediate() will
still return ErrWaitTimeout once the timeout elapsed.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-28 06:56:41 +00:00
Humble Chirammal
b4452f0787 e2e: validate the error return instead of object for consitency
If loadPVC() fails, it return error and we expect the PVC object
to be nil too. In many places we check on the error and exit.
However in few places we are looking at PVC object.
This commit make the condition check on `err` instead of `PVC`
object for consistency.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-10-21 13:56:12 +00:00
Humble Chirammal
dc4da25a30 e2e: correct typo in encryption string
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-10-21 13:56:12 +00:00
Niels de Vos
af922f267a e2e: use k8sVersionGreaterEquals() for rbd upgrade tests
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-19 16:00:39 +00:00
Niels de Vos
a2c077e0d5 e2e: use k8sVersionGreaterEquals() for cephfs upgrade tests
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-19 16:00:39 +00:00
Niels de Vos
cfddc9f887 e2e: use k8sVersionGreaterEquals() for cephfs tests
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-19 16:00:39 +00:00
Niels de Vos
d26030ca5f e2e: use k8sVersionGreaterEquals() for rbd tests
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-19 16:00:39 +00:00
Niels de Vos
c4289f01e8 e2e: add k8sVersionGreaterEquals() helper to check k8s versions
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-19 16:00:39 +00:00
Humble Chirammal
c0eca87268 e2e: make use of new helper function for filesystem mode PVC clone
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-10-15 15:37:14 +00:00
Humble Chirammal
25400a7334 e2e: Introduce e2e test for block mode PVC
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-10-15 15:37:14 +00:00
Humble Chirammal
5763e996af e2e: add validatePVCClone helper function
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-10-15 15:37:14 +00:00
Humble Chirammal
2f828cf20c e2e: update readme for new sidecar repository path
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-10-13 12:34:42 +00:00
Yug
c89afebd4a ci: Upgrade rbd upgrade tests
Add snapshot and pvc cloning test cases
to rbd upgrade testing.

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-09-30 18:07:15 +00:00
Yug
726dcf4da3 ci: Update cephfs upgrade tests
Add snapshot cloning and pvc cloning test cases for
cephfs upgrade tests.

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-09-30 18:07:15 +00:00
Yug
63b5a7ac96 ci: add function to validate checksum
calculateSha512sum() returns the md5sum of a file inside a pod.

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-09-30 18:07:15 +00:00
Nico Berlee
6a5f1380b0 deploy: add pod anti-affinity for provisioner deployments
The added anti-affinity rules prevent provisioner operators from scheduling on
the same nodes. The kubernetes scheduler will spread the pods across nodes to
improve availability during node failures.

Signed-off-by: Nico Berlee <nico.berlee@on2it.net>
2020-09-29 09:29:58 +00:00
Prasanna Kumar Kalever
27c8318a82 doc: update memory assigned to minikube vm
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2020-09-28 16:00:45 +00:00
Madhu Rajanna
dc38320fad e2e: add backend validation for rados namespaced images
added more backend validation for rados namespaced
images.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-09-25 15:35:22 +00:00
Niels de Vos
f41cd105c4 e2e: add simple ceph-fuse StorageClass tests
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-09-23 12:12:51 +00:00
Niels de Vos
85df59531b e2e: pass SC parameters to createCephfsStorageClass()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-09-23 12:12:51 +00:00
Madhu Rajanna
d1f175d9f3 rbd: add support for rbd map and unmap options
added support for providing map and unmap
options to rbd CLI when mapping rbd image
on the node.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-09-21 13:27:28 +00:00