Commit Graph

1647 Commits

Author SHA1 Message Date
Madhu Rajanna
f4c80dec9a Implement NodeStage and NodeUnstage for rbd
in NodeStage RPC call  we  have to map the
device to the node plugin and make  sure  the
the device will be mounted to  the global path

in  nodeUnstage request unmount the device from
global path and unmap the device

if the volume mode is block  we will be creating
a file inside a stageTargetPath  and it will be
considered  as the global path

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-07-24 12:49:21 +00:00
Poornima G
7d8f465746 Temperory workaround for the regression to pass
Currently the cephfs PVC create/delete and all other operations
related to cephfs are failing. One of the recent commits in rook
900abbc967e108ad622648b740a7c57f1268209f has modified ceph-mgr
to run as ceph user rather than root user. The ceph user currently
has no permission to write to the root of the cephfs filesystem.
The fix will be external to CSI itself, but until that lands, sending
a workaround patch so the CSI CI is unblocked

In this patch, we are setting the permission 777 on root of the cephfs
filesystem. Thus ceph user will be able to modify the cephfs filesystem.

Signed-off-by: Poornima G <pgurusid@redhat.com>
2019-07-24 12:04:46 +00:00
Humble Devassy Chirammal
5d5a6c4d91
Merge pull request #469 from Madhu-1/driver-version
Update driver version during build time
2019-07-24 14:41:45 +05:30
Humble Chirammal
0f1bee84ff Add slack invite to the README
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2019-07-17 10:37:46 +00:00
ShyamsundarR
e5e332eded Use correct file descriptor to parse errors
File descriptors in use to parse errors from a few command
invocations were incorrect. This led to inability to detect
certain errors cases and act accordingly.

One of the easiest noticeable issues was when an image is deleted
but its RADOS keys and maps are still intact. In such cases
the DeleteVolume call always errored out unable to find the
image rather than, proceed with cleaning up the RADOS objects
and returning a success.

The original method of using stdout was incorrect, as the command
was tested from within a shell script and the scripts STDIN/OUT/ERR
was redirected to understand behavior. This is now tested using just
the CLI in question, and also examining Ceph code, and further
testing a couple of edge conditions by deleting backing images
for PVs

Signed-off-by: ShyamsundarR <srangana@redhat.com>
2019-07-16 07:51:10 +00:00
Madhu Rajanna
3f8bd3b2a6 Update driver version during build time
update driver version and add git commit
to the image. This will help us to identify
what latest git commit image contains.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-07-12 15:54:52 +05:30
Humble Devassy Chirammal
4b3bf68b37
Merge pull request #468 from humblec/readme
README update for v1.1.0 release
2019-07-12 13:24:16 +05:30
Humble Chirammal
5531c36795 README update for v1.1.0 release
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2019-07-12 13:19:36 +05:30
Poornima G
0d566ee30c Backward compatibility for deleting and mounting old volumes
Signed-off-by: Poornima G <pgurusid@redhat.com>
2019-07-12 05:42:41 +00:00
Poornima G
32ea550e3a Modify CephFs provisioner to use the ceph mgr commands
Currently CephFs provisioner mounts the ceph filesystem
and creates a subdirectory as a part of provisioning the
volume. Ceph now supports commands to provision fs subvolumes,
hance modify the provisioner to use ceph mgr commands to
(de)provision fs subvolumes.

Signed-off-by: Poornima G <pgurusid@redhat.com>
2019-07-12 05:42:41 +00:00
ShyamsundarR
fa68c35f3b Support mounting and deleting version 1.0.0 RBD volumes
This commit adds support to mount and delete volumes provisioned by older
plugin versions (1.0.0) in order to support backward compatibility to 1.0.0
created volumes.

It adds back the ability to specify where older meta data was specified, using
the metadatastorage option to the plugin. Further, using the provided meta data
to mount and delete the older volumes.

It also supports a variety of ways in which monitor information may have been
specified (in the storage class, or in the secret), to keep the monitor
information current.

Testing done:
- Mount/Delete 1.0.0 plugin created volume with monitors in the StorageClass
- Mount/Delete 1.0.0 plugin created volume with monitors in the secret with
  a key "monitors"
- Mount/Delete 1.0.0 plugin created volume with monitors in the secret with
  a user specified key
- PVC creation and deletion with the current version (to ensure at the minimum
  no broken functionality)
- Tested some negative cases, where monitor information is missing in secrets
  or present with a different key name, to understand if failure scenarios work
  as expected

Updates #378

Follow-up work:
- Documentation on how to upgrade to 1.1 plugin and retain above functionality
  for older volumes

Signed-off-by: ShyamsundarR <srangana@redhat.com>
2019-07-08 15:40:17 +00:00
Madhu Rajanna
09f126691c Add nil check for process
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-07-08 12:11:53 +00:00
Humble Devassy Chirammal
3dde768894
Merge pull request #456 from Madhu-1/rbd-mount-opt
support mountflags during nodepublish in rbd
2019-07-08 10:45:07 +05:30
Humble Devassy Chirammal
d1414241dd
Merge pull request #459 from Madhu-1/fix-noexec
Replace noexec with debug mountflag for  cephfs
2019-07-08 10:38:50 +05:30
Madhu Rajanna
306526db97 Replace noexec with debug mountflag for cephfs
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-07-04 14:46:59 +05:30
Madhu Rajanna
df2680c50b support mountflags during nodepublish in rbd
Fixes: #292

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-07-03 21:46:36 +05:30
Madhu Rajanna
5cf0599367 Enable Travis to run E2E against kube 1.15.0
Remove Travis from skip doc check

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-07-02 11:47:05 +00:00
Madhu Rajanna
ffe99403fc update minikube version to v1.2.0
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-07-02 11:47:05 +00:00
ShyamsundarR
c4a3675cec Move locks to more granular locking than CPU count based
As detailed in issue #279, current lock scheme has hash
buckets that are count of CPUs. This causes a lot of contention
when parallel requests are made to the CSI plugin. To reduce
lock contention, this commit introduces granular locks per
identifier.

The commit also changes the timeout for gRPC requests to Create
and Delete volumes, as the current timeout is 10s (kubernetes
documentation says 15s but code defaults are 10s). A virtual
setup takes about 12-15s to complete a request at times, that leads
to unwanted retries of the same request, hence the increased
timeout to enable operation completion with minimal retries.

Tests to create PVCs before and after these changes look like so,

Before:
Default master code + sidecar provisioner --timeout option set
to 30 seconds

20 PVCs
Creation: 3 runs, 396/391/400 seconds
Deletion: 3 runs, 218/271/118 seconds
  - Once was stalled for more than 8 minutes and cancelled the run

After:
Current commit + sidecar provisioner --timeout option set to 30 sec
20 PVCs
Creation: 3 runs, 42/59/65 seconds
Deletion: 3 runs, 32/32/31 seconds

Fixes: #279
Signed-off-by: ShyamsundarR <srangana@redhat.com>
2019-07-01 14:10:14 +00:00
ShyamsundarR
bc39c523b7 Fix returning success from DeleteSnapshot for stale requests
Also reduced code duplication in fetching pool list from Ceph.

DeleteSnapshot like DeleteVolume, should return a success when it
detects that the snapshot keys are missing from the RADOS OMaps that
store the snapshot UUID to request name mapping.

This was missing in the code, and is now added.

Signed-off-by: ShyamsundarR <srangana@redhat.com>
2019-07-01 10:54:53 +00:00
james58899
1e8fa38879 Fix nil err cause nbd fail to mount 2019-06-29 11:12:58 +00:00
Humble Chirammal
027331c186 Use sidecar which support cloning
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2019-06-28 01:11:06 +00:00
Madhu Rajanna
59d3365d3b update statefulset and daemonset api-version
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-06-25 14:00:46 +00:00
Humble Chirammal
b82d75e846 Revert "Update README.md"
This reverts commit cf426fff25.
2019-06-24 15:57:58 +00:00
Humble Devassy Chirammal
cf426fff25
Update README.md 2019-06-24 20:36:34 +05:30
ShyamsundarR
c5762b6b5c Modify RBD plugin to use a single ID and move the id and key into the secret
RBD plugin needs only a single ID to manage images and operations against a
pool, mentioned in the storage class. The current scheme of 2 IDs is hence not
needed and removed in this commit.

Further, unlike CephFS plugin, the RBD plugin splits the user id and the key
into the storage class and the secret respectively. Also the parameter name
for the key in the secret is noted in the storageclass making it a variant and
hampers usability/comprehension. This is also fixed by moving the id and the key
to the secret and not retaining the same in the storage class, like CephFS.

Fixes #270

Testing done:
- Basic PVC creation and mounting

Signed-off-by: ShyamsundarR <srangana@redhat.com>
2019-06-24 13:46:14 +00:00
Humble Chirammal
22ff5c0911 Migrate from snapClient.VolumesnapshotV1alpha1Client to
snapClient.SnapshotV1alpha1Client and also update kube dependency

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2019-06-24 13:08:29 +00:00
Humble Chirammal
3bc6771df8 Migrate from framwork.Logf and also use new nsenter interface
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2019-06-24 13:08:29 +00:00
Madhu Rajanna
b3b181f2a5 skip pvc raw block test for travis CI
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-06-23 11:53:37 +00:00
Madhu Rajanna
2e1fc352b3 Add validation of backend rbd image and snapshots
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-06-23 11:53:37 +00:00
Madhu Rajanna
a083cb713f update golang version to 1.12.x
tls.VersionTLS13 requires 1.12.x

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-06-23 11:53:37 +00:00
Madhu Rajanna
1a2d71e580 Add code to test snapshot,pvc clone and pvc block mode
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-06-23 11:53:37 +00:00
Madhu Rajanna
e32b649648 Add external-snapshotter to ceph-csi
Adding extenal-provisoner will help us to
test snapshot functionality

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-06-23 11:53:37 +00:00
Madhu Rajanna
7f993707ca rbd supports ReadWriteMany for block mode
update example rbd PVC from ReadWriteMany
to ReadWriteOnce as rbd supports ReadWriteMany
only if pvc mode is block

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-06-18 04:41:01 +00:00
Madhu Rajanna
983f28ad2f Revert "Use Deployment with leader election instead of StatefulSet"
This reverts commit a151bec94b.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-06-14 13:39:03 +00:00
Madhu Rajanna
fb0cbef68b update travis to run func test against different kube version
updated travis to run functional tests
against different kubernetes versions

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-06-14 13:18:24 +00:00
Madhu Rajanna
69662e63ed change permission of targetpath
setting the permission of  targetpath to 777
will allow non-root user to write to pv.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-06-12 14:24:37 +00:00
Humble Devassy Chirammal
9d27b6b425
Merge pull request #422 from humblec/travis-1
Use v1.1.1 of minikube
2019-06-11 12:59:36 +05:30
Humble Chirammal
8fb87ef8a4 Use v1.1.1 of minikube
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2019-06-11 12:31:05 +05:30
Humble Devassy Chirammal
5b926ec72c
Merge pull request #417 from Madhu-1/update-helm
update helm chart version
2019-06-10 20:48:49 +05:30
Madhu Rajanna
bccfafdfb2 update helm chart version
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-06-10 16:54:05 +05:30
Humble Devassy Chirammal
46b19a93f5
Merge pull request #416 from Madhu-1/update-golanglint
Enable all static-checks in golangci-lint
2019-06-10 16:49:40 +05:30
Humble Devassy Chirammal
dab551c8e2
Merge pull request #419 from humblec/release-prep
Add pointers and sidecar information in the README
2019-06-10 16:48:03 +05:30
Madhu Rajanna
a38986fce0 Enable all static-checks in golangci-lint
* Enable all static-checks in golangci-lint
* Update golangci-lint version
* Fix issue found in golangci-lint

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-06-10 15:56:17 +05:30
Humble Chirammal
9b384f38e6 Add pointers and sidecar information in the README
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2019-06-10 15:21:01 +05:30
Humble Devassy Chirammal
5753325970
Merge pull request #418 from humblec/e2e
Combine the variable section in e2e and also introuce new vars.
2019-06-10 14:10:24 +05:30
Humble Chirammal
66bb0e4e54 Combine the variable section in e2e and also introuce new vars.
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2019-06-10 13:50:30 +05:30
Humble Devassy Chirammal
47d4e44af0
Merge pull request #414 from Madhu-1/fix_335
Use Deployment with leader election instead of StatefulSet
2019-06-10 13:14:20 +05:30
Humble Devassy Chirammal
4df939793b
Merge pull request #372 from Madhu-1/fix-err
Fix error string as per golang standard
2019-06-10 13:02:07 +05:30
Madhu Rajanna
7d3a6105c7 Fix misspell words
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-06-10 12:52:13 +05:30