Commit Graph

118 Commits

Author SHA1 Message Date
Niels de Vos
d26030ca5f e2e: use k8sVersionGreaterEquals() for rbd tests
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-19 16:00:39 +00:00
Humble Chirammal
c0eca87268 e2e: make use of new helper function for filesystem mode PVC clone
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-10-15 15:37:14 +00:00
Humble Chirammal
25400a7334 e2e: Introduce e2e test for block mode PVC
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-10-15 15:37:14 +00:00
Nico Berlee
6a5f1380b0 deploy: add pod anti-affinity for provisioner deployments
The added anti-affinity rules prevent provisioner operators from scheduling on
the same nodes. The kubernetes scheduler will spread the pods across nodes to
improve availability during node failures.

Signed-off-by: Nico Berlee <nico.berlee@on2it.net>
2020-09-29 09:29:58 +00:00
Madhu Rajanna
dc38320fad e2e: add backend validation for rados namespaced images
added more backend validation for rados namespaced
images.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-09-25 15:35:22 +00:00
Madhu Rajanna
d1f175d9f3 rbd: add support for rbd map and unmap options
added support for providing map and unmap
options to rbd CLI when mapping rbd image
on the node.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-09-21 13:27:28 +00:00
Madhu Rajanna
b4693dcffe e2e: rework on E2E framework
rework of E2E framework for better code
organization and add more helpful logs for
debugging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-09-10 07:40:13 +00:00
Madhu Rajanna
d5c19954c1 e2e: add backend image validation for rbd
validate backend rbd images count in each
E2E test cases. This helps a lot to catch
the issues in each test case.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-24 08:16:59 +00:00
Mehdy Khoshnoody
204487383d e2e: Add e2e test cases for rbd rados namespace
These test cases are will be executed against a rados namespace.
- Create a PVC and bind it to an app.
- Resize block PVC and check device size.
- Create a PVC clone and bind it to an app.

Signed-off-by: Mehdy Khoshnoody <mehdy.khoshnoody@gmail.com>
2020-08-12 16:22:58 +05:30
Yug
02b4a7175c rbd: add upgrade testing
Upgrade testing will enable us to keep
in check the backward compatibility of
earlier releases.

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-07-26 03:41:53 +00:00
Madhu Rajanna
5168ad7ddf e2e: create/delete snap and clone in parallel
In rbd E2E testing,we need to create snap and clone
as parallel operation.

This helps us to insure that functionality works when
we have parallel delete and create operations and also
it helps to catch bugs when we get parallel requests.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-21 13:25:19 +00:00
Madhu Rajanna
9077c25c15 e2e: add e2e to test pvc-pvc cloning
Added an e2e testcase to test pvc-pvc
cloning.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-10 14:02:12 +00:00
Madhu Rajanna
63b795d165 e2e: provide an option to skip rbd and cephfs e2e
Added two new parameters for e2e test to skip
rbd and cephfs tests. This will help us to
run more test in Travis CI.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-09 06:42:28 +00:00
Madhu Rajanna
6838d30aff e2e: reduce the pod cound and wait time
Reduced the number of pods created
in ROX E2E to save some time in E2E
and changed the waiting time from 2 to 1
min.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-06 10:08:31 +00:00
Madhu Rajanna
47fb5f2299 e2e: update E2E testing for snapshot
with new implemntation when user creates a snapshot
in backend we are creating rbd image, we need to
validate the total images count in backend when
creating snapshots and clones.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-01 08:21:47 +00:00
Madhu Rajanna
39561b9675 e2e: Add E2E to mount rbd PVC as ro in app
Added an E2E to mount rbd PVC as readonly
in application pod and try to create some
file in Readonly PVC,when we try to create
files on RO PVC, we should get error.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-06-22 06:15:40 +00:00
Madhu Rajanna
649aeb7aaf rbd: Add support for rbd ROX PVC mounting
if the PVC access mode is ReadOnlyMany
or single node readonly, mounting the rbd
device path to the staging path as readonly
to avoid the write operation.

If the PVC acccess mode is readonly, mapping
rbd images as readonly.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-06-22 06:15:40 +00:00
Madhu Rajanna
44da7ffb4e e2e: Add testcase of invalid mount option
Added a test case to cover unmap of rbd image
if the mounting fails. if we pass the invalid
mount option the expectation is that mounting
of rbd image to stagingpath fails. as the unmap
happens it should not block the rbd pvc deletion
saying rbd image is in-use.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-06-01 11:27:25 +00:00
Madhu Rajanna
b0270ced41 e2e: reword sentences in e2e
as kube is the shortform for kubernetes.
its expected to mention full form kubernetes
in the e2e tests.

Updated few wordings in the e2e.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-05-20 06:37:36 +00:00
Madhu Rajanna
f0b3cee94a e2e: define replicapool as constant
the static check is failing as the replicapool
is used in 3 or more places, we need to define
a variable and use it.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-05-15 08:19:32 +00:00
Madhu Rajanna
1f13692000 ci: skip snapshot E2E if kube<1.17+
snapshot beta CRD wont work if the
kubernetes version is less than 1.17.0
as the snapshot CRD wont be installed
we cannot test the snapshot,so disabling
it if the kube version is less than 1.17

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-05-15 08:19:32 +00:00
Madhu Rajanna
c5bf95b625 e2e: Enable E2E for rbd snapshot
Updated E2E to use beta client and beta
Snapshot resources.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-05-15 08:19:32 +00:00
ShyamsundarR
1a8f8e3c24 Add support for erasure coded pools
This commit adds support to mention dataPool parameter for the
topology constrained pools in the StorageClass, that can be
leveraged to mention erasure coded pool names to use for RBD
data instead of the replica pools.

Signed-off-by: ShyamsundarR <srangana@redhat.com>
2020-04-14 14:14:29 +00:00
ShyamsundarR
e73921f268 Add e2e tests for topology based provisioning
- This commit adds tests only for RBD, as CephFS still needs
an enhancement in CephFS subvolume commands to effectively use
topology based provisioning

Signed-off-by: ShyamsundarR <srangana@redhat.com>
2020-04-14 14:14:29 +00:00
ShyamsundarR
5c4abf8347 Add topology support to ceph-csi
Signed-off-by: ShyamsundarR <srangana@redhat.com>
2020-04-14 14:14:29 +00:00
Humble Chirammal
4c96ad3c85 Changes to e2e to accomodate client-go changes and RunKubectlInput
With client-go v1.18.0 there is a change where Signatures on methods
in generated clientsets, dynamic, metadata, and scale clients have been
modified to accept context.Context as a first argument.
Signatures of Create, Update, and Patch methods have been updated to accept
CreateOptions, UpdateOptions and PatchOptions respectively.
Signatures of Delete and DeleteCollection methods now accept DeleteOptions
by value instead of by reference

The framework.RunkubectlInput now accepts namespace as the first parameter
which is also accommodated with this PR.

Signed-off-by: Humble Chirammal hchiramm@redhat.com
2020-04-14 10:50:12 +00:00
Madhu Rajanna
d09ffbd6de helm: add helm charts E2E
This PR adds the support for helm
installation, and cephcsi helm charts
deployment and teardown and also runs E2E
on for helm charts.

Add socat to provide port forwadring access for helm

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-04-06 11:01:25 +00:00
Madhu Rajanna
e788328750 Refractor E2E to reduce code duplication
Updated E2E to reduce code duplication
and create resouces in different namespaces.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-03-20 10:16:23 +00:00
Madhu Rajanna
c45c426215 Add cephcsi namespace and rook namespace flag
Added namespace flag to cephcsi to deploy cephcsi
resouces in different namespace.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-03-20 10:16:23 +00:00
Madhu Rajanna
59fe8c1f2f Provide option to skip ceph-csi plugin deployment
To test helm charts in CI we need to skip the ceph-csi
deployment in E2E, This PR provides an option in E2E
to enable/disable cephcsi deployment.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-03-20 10:16:23 +00:00
Madhu Rajanna
b4e6504e9b Fix mountoption issue in rbd
use mountoptions when mounting rbd to stagingpath
in stagevolume request, add E2E for mount options

fixes: #846
updates: #757

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-03-13 14:33:56 +00:00
Reinier Schoof
a4532fafd0 added volumeNamePrefix and snapshotNamePrefix as parameters for storageClass
this allows administrators to override the naming prefix for both volumes and snapshots
created by the rbd plugin.

Signed-off-by: Reinier Schoof <reinier@skoef.nl>
2020-02-25 05:03:51 +00:00
Madhu Rajanna
1ec2bc47e3 Add E2E for rbd static PVC
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-02-18 12:34:46 +00:00
Madhu Rajanna
8dcb6a6105 Handle Delete operation if pool not found
If the backend rbd or cephfs pool is already deleted
we need to return success to the  DeleteVolume RPC
call to make it idempotent.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-02-11 15:58:15 +00:00
Vasyl Purchel
669dc4536f Reduce encryption KMS configuration SC parameters
* moves KMS type from StorageClass into KMS configuration itself
 * updates omapval used to identify KMS to only it's ID without the type

why?

1. when using multiple KMS configurations (not currently supported)
automated parsing of kms configuration will be failing because some
entries in configs won't comply with the requested type
2. less options are needed in the StorageClass and less data used to
identify the KMS

Signed-off-by: Vasyl Purchel vasyl.purchel@workday.com
Signed-off-by: Andrea Baglioni andrea.baglioni@workday.com
2020-02-10 15:21:11 +00:00
Vasyl Purchel
419ad0dd8e Adds per volume encryption with Vault integration
- adds proposal document for PVC encryption from PR448
- adds per-volume encription by generating encryption passphrase
  for each volume and storing it in a KMS
- adds HashiCorp Vault integration as a KMS for encryption passphrases
- avoids encrypting volume second time if it was already encrypted but
  no file system created
- avoids unnecessary checks if volume is a mapped device when encryption
  was not requested
- prevents resizing encrypted volumes (it is not currently supported)
- prevents creating snapshots from encrypted volumes to prevent attack
  on encryption key (security guard until re-encryption of volumes
  implemented)

Signed-off-by: Vasyl Purchel vasyl.purchel@workday.com
Signed-off-by: Andrea Baglioni andrea.baglioni@workday.com

Fixes #420
Fixes #744
2020-02-05 05:18:56 +00:00
Oguz Kilcan
aadce54b2f Added PodSecurityPolicy support 2020-01-22 08:19:42 +00:00
wilmardo
f04af5742d refact: Remove Kubernetes 1.13.x support
Signed-off-by: wilmardo <info@wilmardenouden.nl>
2020-01-20 10:32:30 +00:00
Humble Chirammal
2d0ed298e3 Update framework imports and its functions.
Signed-off-by: Cedric Hauber <hauber.c@gmail.com>
2020-01-17 12:06:02 +00:00
Madhu Rajanna
ad7e5ca301 Add E2E for block PVC resize
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-12-31 06:10:40 +00:00
Madhu Rajanna
7d15992769 Enable logging in E2E if test fails
source: https://github.com/kubernetes/kubernetes/blob/master/test/e2e/framework/kubectl/kubectl_utils.go
kubectlLogPod function is not exposed in above code so copied it.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-12-18 12:35:17 +00:00
Madhu Rajanna
b4f20e941c Enable Block E2E for rbd
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-12-18 12:35:17 +00:00
Vasyl Purchel
166eaf700f Adds PVC encryption with LUKS
Adds encryption in StorageClass as a parameter. Encryption passphrase is
stored in kubernetes secrets per StorageClass. Implements rbd volume
encryption relying on dm-crypt and cryptsetup using LUKS extension

The change is related to proposal made earlier. This is a first part of
the full feature that adds encryption with passphrase stored in secrets.

Signed-off-by: Vasyl Purchel vasyl.purchel@workday.com
Signed-off-by: Andrea Baglioni andrea.baglioni@workday.com
Signed-off-by: Ioannis Papaioannou ioannis.papaioannou@workday.com
Signed-off-by: Paul Mc Auley paul.mcauley@workday.com
Signed-off-by: Sergio de Carvalho sergio.carvalho@workday.com
2019-12-16 08:12:44 +00:00
Humble Chirammal
7c8e66e427 Add resize check for XFS formatted FS
Lock out parellel requests against same volumeID
Remove pod after resize and validation in E2E

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2019-12-13 12:40:12 +00:00
Humble Chirammal
e69f462336 Add e2e tests for RBD resizer
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2019-12-13 12:40:12 +00:00
Woohyung Han
5c96ab1d0e Fix to use kubectl create not kubectl apply in the e2e
Signed-off-by: Woohyung Han <techhanx@gmail.com>
2019-12-11 09:31:03 +00:00
Woohyung Han
2c9d711463 Move rook-deploy code from e2e to ./scripts/minikube.sh
We have the e2e test with --deploy-rook=true that makes all test
environment. It works fine, but It does not seem to be the role of
e2e test. In addition, when developing the code we need to run full
test scenario with deploying rook every time, or we need to build
rook environment by hand. Move rook-deploy code to minikube.sh.
2019-12-05 12:32:59 +00:00
Woohyung Han
685e2540a8 Fix typo in deleteConfigMap 2019-12-04 04:31:50 +00:00
Madhu Rajanna
69b7c4672f Added E2E for rbd nodeplugin restart
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-09-23 13:22:29 +00:00
Humble Chirammal
0e6617e1ff Format RBD volumes with nodiscard formatting options.
Currently rbd CSI plugin uses formatAndMount of
mount.SafeFormatAndMount. This does not allow to pass or use
specific formatting arguments with it. This patch introduce
RBD specific formatting options with both xfs and ext4,
for example: -E no-discard with ext4 and -k option with
XFS to boost formatting performance of RBD device.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2019-09-20 13:16:53 +00:00
Madhu Rajanna
64ca401a51 Fix mon endpoint issue in E2E
in toolbox mon endpoints are not
updated properly, this is causing an issue in E2E
this PR is a workaround to fix this issue.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-09-04 11:12:07 +00:00
Madhu Rajanna
ae67534a44 Add wait logic to check fsid from toolbox
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-08-30 06:50:32 +00:00
Madhu Rajanna
90fef919d5 Skip snapshot testing in CI
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-08-13 16:22:55 +05:30
Madhu Rajanna
4ed187eb3f Fix issues in E2E cleanup
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-08-07 20:26:20 +05:30
Madhu Rajanna
02bcb5f16a Enable leader election in v1.14+
Use Deployment with leader election instead of StatefulSet

Deployment behaves better when a node gets disconnected
from the rest of the cluster - new provisioner leader
is elected in ~15 seconds, while it may take up to
5 minutes for StatefulSet to start a new replica.

Refer: kubernetes-csi/external-provisioner@52d1fbc

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-08-05 07:11:44 +00:00
Madhu Rajanna
f4c80dec9a Implement NodeStage and NodeUnstage for rbd
in NodeStage RPC call  we  have to map the
device to the node plugin and make  sure  the
the device will be mounted to  the global path

in  nodeUnstage request unmount the device from
global path and unmap the device

if the volume mode is block  we will be creating
a file inside a stageTargetPath  and it will be
considered  as the global path

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-07-24 12:49:21 +00:00
Humble Chirammal
3bc6771df8 Migrate from framwork.Logf and also use new nsenter interface
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2019-06-24 13:08:29 +00:00
Madhu Rajanna
b3b181f2a5 skip pvc raw block test for travis CI
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-06-23 11:53:37 +00:00
Madhu Rajanna
2e1fc352b3 Add validation of backend rbd image and snapshots
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-06-23 11:53:37 +00:00
Madhu Rajanna
1a2d71e580 Add code to test snapshot,pvc clone and pvc block mode
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-06-23 11:53:37 +00:00
Madhu Rajanna
983f28ad2f Revert "Use Deployment with leader election instead of StatefulSet"
This reverts commit a151bec94b.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-06-14 13:39:03 +00:00
Madhu Rajanna
69662e63ed change permission of targetpath
setting the permission of  targetpath to 777
will allow non-root user to write to pv.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-06-12 14:24:37 +00:00
Humble Devassy Chirammal
46b19a93f5
Merge pull request #416 from Madhu-1/update-golanglint
Enable all static-checks in golangci-lint
2019-06-10 16:49:40 +05:30
Madhu Rajanna
a38986fce0 Enable all static-checks in golangci-lint
* Enable all static-checks in golangci-lint
* Update golangci-lint version
* Fix issue found in golangci-lint

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-06-10 15:56:17 +05:30
Humble Chirammal
66bb0e4e54 Combine the variable section in e2e and also introuce new vars.
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2019-06-10 13:50:30 +05:30
Madhu Rajanna
a151bec94b Use Deployment with leader election instead of StatefulSet
Deployment behaves better when a node gets disconnected from the rest of
the cluster - new provisioner leader is elected in ~15 seconds, while
it may take up to 5 minutes for StatefulSet to start a new replica.

Refer: 52d1fbcf9d

Fixes: #335

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-06-10 09:51:22 +05:30
Madhu Rajanna
a4655f7549 create config map for cephfs
add fs name to storageclass in e2e

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-06-07 15:38:23 +05:30
Madhu Rajanna
fc529f4831 Add E2E framework
deploys rook
deploy cephfs plugin
deploy rbd plugin
create PVC/POD to test cephfs
create PVC/POD to test rbd

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-06-04 11:39:42 +05:30