Commit Graph

64 Commits

Author SHA1 Message Date
Madhu Rajanna
6da96c6327 remove support for create image with image-format 1
tried to create an image with image-format=1

```
sh-4.2# rbd create --size=1024 replicapool/test --image-format=1
rbd: image format 1 is deprecated
rbd: create error: (22) Invalid argument
2019-09-11 07:00:54.531 7fb0e40bfb00 -1 librbd: Format 1 image creation unsupported.
```

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-09-12 07:52:32 +00:00
Madhu Rajanna
41b701c98c Add support for erasure pool in rbd
Allow specifying different metadata and data pools in a
CSI RBD StorageClass

Fixes: #199
Fixes: https://github.com/rook/rook/issues/2650
Fixes: https://github.com/rook/rook/issues/3763

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-09-11 06:48:08 +00:00
Niels de Vos
dd668e59f1 Address security concerns reported by 'gosec'
gosec reports several issues, none of them looks very critical. With
this change the following concerns have been addressed:

[pkg/cephfs/nodeserver.go:229] - G302: Expect file permissions to be 0600 or less (Confidence: HIGH, Severity: MEDIUM)
  > os.Chmod(targetPath, 0777)

[pkg/cephfs/util.go:39] - G204: Subprocess launched with variable (Confidence: HIGH, Severity: MEDIUM)
  > exec.Command(program, args...)

[pkg/rbd/nodeserver.go:156] - G302: Expect file permissions to be 0600 or less (Confidence: HIGH, Severity: MEDIUM)
  > os.Chmod(stagingTargetPath, 0777)

[pkg/rbd/nodeserver.go:205] - G302: Expect file permissions to be 0600 or less (Confidence: HIGH, Severity: MEDIUM)
  > os.OpenFile(mountPath, os.O_CREATE|os.O_RDWR, 0750)

[pkg/rbd/rbd_util.go:797] - G304: Potential file inclusion via variable (Confidence: HIGH, Severity: MEDIUM)
  > ioutil.ReadFile(fPath)

[pkg/util/cephcmds.go:35] - G204: Subprocess launched with variable (Confidence: HIGH, Severity: MEDIUM)
  > exec.Command(program, args...)

[pkg/util/credentials.go:47] - G104: Errors unhandled. (Confidence: HIGH, Severity: LOW)
  > os.Remove(tmpfile.Name())

[pkg/util/credentials.go:92] - G104: Errors unhandled. (Confidence: HIGH, Severity: LOW)
  > os.Remove(cr.KeyFile)

[pkg/util/pidlimit.go:74] - G304: Potential file inclusion via variable (Confidence: HIGH, Severity: MEDIUM)
  > os.Open(pidsMax)

URL: https://github.com/securego/gosec
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2019-09-04 11:48:37 +00:00
Daniel-Pivonka
01a78cace5 switch to cephfs, utils, and csicommon to new loging system
Signed-off-by: Daniel-Pivonka <dpivonka@redhat.com>
2019-08-29 14:04:31 +00:00
Madhu Rajanna
3af364e7b5 move to statand context package
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-08-26 06:19:24 +00:00
Madhu Rajanna
38ca08bf65 Context based logging for rbd
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-08-26 06:19:24 +00:00
Madhu Rajanna
2ca575b99d Wrap error if failed to fetch mon
This will help user to check whats
the actual error. if the config file
is having issue or the  clusterid is
not valid.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-08-13 17:16:27 +00:00
ShyamsundarR
20d336fca3 Add support to use ceph manager rbd command to delete an image
Image deletion takes time proportional to the size of the
image. Hence, ceph manager is enhanced to support async
deletion of an image, or rather passing the task of
deleting an image to the ceph manager.

This commit leverages the ceph manager enhancement in the CSI code.

NOTE: This is tested against a ceph cluster that is running
Ceph master version of the code. Once other releases
catch up in terms of the feature, the optimization would be
available to the CSI driver as well.

Fixes: #523
Signed-off-by: ShyamsundarR <srangana@redhat.com>
2019-08-13 16:08:22 +00:00
ShyamsundarR
885ec7049d Update Unstage transaction to undo steps done in Stage
In unstage we now adhere to the transaction (or order of steps)
done in Stage. To enable this we stash the image meta data
into a local file on the staging path for use with unstage
request.

This helps in unmapping a stale map, in case the mount or
other steps in the transaction are complete.

Signed-off-by: ShyamsundarR <srangana@redhat.com>
2019-08-13 14:07:52 +00:00
ShyamsundarR
bd204d7d45 Use --keyfile option to pass keys to all Ceph CLIs
Every Ceph CLI that is invoked at present passes the key via the
--key option, and hence is exposed to key being displayed on
the host using a ps command or such means.

This commit addresses this issue by stashing the key in a tmp
file, which is again created on a tmpfs (or empty dir backed by
memory). Further using such tmp files as arguments to the --keyfile
option for every CLI that is invoked.

This prevents the key from being visible as part of the argument list
of the invoked program on the system.

Fixes: #318

Signed-off-by: ShyamsundarR <srangana@redhat.com>
2019-07-25 12:46:15 +00:00
Madhu Rajanna
f4c80dec9a Implement NodeStage and NodeUnstage for rbd
in NodeStage RPC call  we  have to map the
device to the node plugin and make  sure  the
the device will be mounted to  the global path

in  nodeUnstage request unmount the device from
global path and unmap the device

if the volume mode is block  we will be creating
a file inside a stageTargetPath  and it will be
considered  as the global path

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-07-24 12:49:21 +00:00
ShyamsundarR
e5e332eded Use correct file descriptor to parse errors
File descriptors in use to parse errors from a few command
invocations were incorrect. This led to inability to detect
certain errors cases and act accordingly.

One of the easiest noticeable issues was when an image is deleted
but its RADOS keys and maps are still intact. In such cases
the DeleteVolume call always errored out unable to find the
image rather than, proceed with cleaning up the RADOS objects
and returning a success.

The original method of using stdout was incorrect, as the command
was tested from within a shell script and the scripts STDIN/OUT/ERR
was redirected to understand behavior. This is now tested using just
the CLI in question, and also examining Ceph code, and further
testing a couple of edge conditions by deleting backing images
for PVs

Signed-off-by: ShyamsundarR <srangana@redhat.com>
2019-07-16 07:51:10 +00:00
ShyamsundarR
fa68c35f3b Support mounting and deleting version 1.0.0 RBD volumes
This commit adds support to mount and delete volumes provisioned by older
plugin versions (1.0.0) in order to support backward compatibility to 1.0.0
created volumes.

It adds back the ability to specify where older meta data was specified, using
the metadatastorage option to the plugin. Further, using the provided meta data
to mount and delete the older volumes.

It also supports a variety of ways in which monitor information may have been
specified (in the storage class, or in the secret), to keep the monitor
information current.

Testing done:
- Mount/Delete 1.0.0 plugin created volume with monitors in the StorageClass
- Mount/Delete 1.0.0 plugin created volume with monitors in the secret with
  a key "monitors"
- Mount/Delete 1.0.0 plugin created volume with monitors in the secret with
  a user specified key
- PVC creation and deletion with the current version (to ensure at the minimum
  no broken functionality)
- Tested some negative cases, where monitor information is missing in secrets
  or present with a different key name, to understand if failure scenarios work
  as expected

Updates #378

Follow-up work:
- Documentation on how to upgrade to 1.1 plugin and retain above functionality
  for older volumes

Signed-off-by: ShyamsundarR <srangana@redhat.com>
2019-07-08 15:40:17 +00:00
ShyamsundarR
c4a3675cec Move locks to more granular locking than CPU count based
As detailed in issue #279, current lock scheme has hash
buckets that are count of CPUs. This causes a lot of contention
when parallel requests are made to the CSI plugin. To reduce
lock contention, this commit introduces granular locks per
identifier.

The commit also changes the timeout for gRPC requests to Create
and Delete volumes, as the current timeout is 10s (kubernetes
documentation says 15s but code defaults are 10s). A virtual
setup takes about 12-15s to complete a request at times, that leads
to unwanted retries of the same request, hence the increased
timeout to enable operation completion with minimal retries.

Tests to create PVCs before and after these changes look like so,

Before:
Default master code + sidecar provisioner --timeout option set
to 30 seconds

20 PVCs
Creation: 3 runs, 396/391/400 seconds
Deletion: 3 runs, 218/271/118 seconds
  - Once was stalled for more than 8 minutes and cancelled the run

After:
Current commit + sidecar provisioner --timeout option set to 30 sec
20 PVCs
Creation: 3 runs, 42/59/65 seconds
Deletion: 3 runs, 32/32/31 seconds

Fixes: #279
Signed-off-by: ShyamsundarR <srangana@redhat.com>
2019-07-01 14:10:14 +00:00
ShyamsundarR
c5762b6b5c Modify RBD plugin to use a single ID and move the id and key into the secret
RBD plugin needs only a single ID to manage images and operations against a
pool, mentioned in the storage class. The current scheme of 2 IDs is hence not
needed and removed in this commit.

Further, unlike CephFS plugin, the RBD plugin splits the user id and the key
into the storage class and the secret respectively. Also the parameter name
for the key in the secret is noted in the storageclass making it a variant and
hampers usability/comprehension. This is also fixed by moving the id and the key
to the secret and not retaining the same in the storage class, like CephFS.

Fixes #270

Testing done:
- Basic PVC creation and mounting

Signed-off-by: ShyamsundarR <srangana@redhat.com>
2019-06-24 13:46:14 +00:00
Humble Devassy Chirammal
95252dd9f6
Merge pull request #390 from ShyamsundarR/stateless-cephfs
Make CephFS plugin stateless reusing RADOS based journal scheme
2019-06-07 10:44:18 +05:30
ShyamsundarR
9a03d735a2 Remove redundant pool parameter from snapshot class
The SnapshotClass for RBD requires a pool parameter. This is redundant
as a snapshot is not created on a different pool than the source image
of the snapshot (refer rbd man page).

Further, when a snapshot needs to be created its source CSI VolumeID
is passed to the creation call, and hence the source volumes pool
needs to be reused to create the snapshot.

Similarly to clone a snapshot, the create request would come in with a
SnapshotID to help identify the snapshot pool, and the same create
request parameters would contain the storage class based pool parameter
to create the clone into (as clones can be in different pools as
compared to their parent snapshots).

Thus, the parameter pool seems redundant in the snapshot class and
should be removed to improve ease of use.

Fixes #379

Signed-off-by: ShyamsundarR <srangana@redhat.com>
2019-06-04 08:34:21 +00:00
ShyamsundarR
b9cd0e18ad Make CephFS plugin stateless reusing RADOS based journal scheme
This is a part of the stateless set of commits for CephCSI.

This commit removes the dependency on config maps to store cephFS provisioned
volumes, and instead relies on RADOS based objects and keys, and required
CSI VolumeID encoding to detect the provisioned volumes.

Changes:
- Provide backward compatibility to provisioned volumes by older plugin versions (1.0.0 or older)
- Remove Create/Delete support for statically provisioned volumes (fixes #382)
- Added namespace support to RADOS OMaps and used the same to store RADOS CSI objects and keys in the CephFS metadata pool
- Added support to mention fsname for CephFS provisioning (fixes #359)
- Changed field name in CSI Identifier to 'location', to denote a pool or fscid
- Updated mounter cache to use new scheme
- Required Helm manifests are updated
- Required documentation and other manifests are updated
- Made driver option 'metadatastorage' as optional, as fresh installs do not need to specify the same

Testing done:
- Create/Mount/Delete PVC
- Create/Delete 5 PVCs
- Mount version 1.0.0 PVC
- Delete version 1.0.0 PV
- Mount Statically defined PV/PVC/Pod
- Mount Statically defined version 1.0.0 PV/PVC/Pod
- Delete Statically defined version 1.0.0 PV/PVC/Pod
- Node restart when mounted to test mountcache
- Use InstanceID other than 'default'
- RBD basic round of tests, as namespace is added to OMaps
- csitest against ceph-fs plugin
  - NOTE: CephFS plugin still does not detect and address already created
  volumes but of a different size
- Test not providing any value to the metadata storage parameter

Signed-off-by: ShyamsundarR <srangana@redhat.com>
2019-05-30 06:20:35 -04:00
ShyamsundarR
1406f29dcd Refactor voljournal to aid reuse with CephFS
and to also inmprove the code reuse in rbd itself.

Signed-off-by: ShyamsundarR <srangana@redhat.com>
2019-05-30 09:58:40 +00:00
ShyamsundarR
d02e50aa9b Removed config maps and replaced with rados omaps
Existing config maps are now replaced with rados omaps that help
store information regarding the requested volume names and the rbd
image names backing the same.

Further to detect cluster, pool and which image a volume ID refers
to, changes to volume ID encoding has been done as per provided
design specification in the stateless ceph-csi proposal.

Additional changes and updates,
- Updated documentation
- Updated manifests
- Updated Helm chart
- Addressed a few csi-test failures

Signed-off-by: ShyamsundarR <srangana@redhat.com>
2019-05-19 12:29:33 +00:00
Madhu Rajanna
f60a07ae82 update vendor to latest kubernetes 1.14.0
some of the kubernetes independent
packages are moved out of the tree to
new projects.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-05-14 06:56:56 +00:00
Humble Chirammal
1eff2e1490 Merge branch 'master' of http://github.com/ceph/ceph-csi into csi-v1.0
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2019-05-07 15:14:14 +05:30
Dylan Redding
e0a1661bee linter fixes 2019-04-29 09:58:04 +00:00
Dylan Redding
b488a5ae85 Fix loading data from configmaps. 2019-04-29 09:58:04 +00:00
wilmardo
891daa9375 Replaces the references to the Kubernete Authors with the Ceph-CSI authors 2019-04-03 11:14:08 +02:00
ShyamsundarR
ba2e5cff51 Address remenant subject reference and code style reviews
Signed-off-by: ShyamsundarR <srangana@redhat.com>
2019-03-26 16:19:24 +00:00
ShyamsundarR
fc0cf957be Updated code and docs to reflect correct terminology
- Updated instances of fsid with clusterid
- Updated instances of credentials/subject with user/key

Signed-off-by: ShyamsundarR <srangana@redhat.com>
2019-03-26 16:19:24 +00:00
ShyamsundarR
e1c685ef39 Fixed scope of confStore
Signed-off-by: ShyamsundarR <srangana@redhat.com>
2019-03-26 16:19:24 +00:00
ShyamsundarR
2064e674a4 Addressed using k8s client APIs to fetch secrets
Based on the review comments addressed the following,
- Moved away from having to update the pod with volumes
when a new Ceph cluster is added for provisioning via the
CSI driver

- The above now used k8s APIs to fetch secrets
  - TBD: Need to add a watch mechanisim such that these
secrets can be cached and updated when changed

- Folded the Cephc configuration and ID/key config map
and secrets into a single secret

- Provided the ability to read the same config via mapped
or created files within the pod

Tests:
- Ran PV creation/deletion/attach/use using new scheme
StorageClass
- Ran PV creation/deletion/attach/use using older scheme
to ensure nothing is broken
- Did not execute snapshot related tests

Signed-off-by: ShyamsundarR <srangana@redhat.com>
2019-03-26 16:19:24 +00:00
ShyamsundarR
97f8c4b677 Provide options to pass in Ceph cluster-id
This commit provides the option to pass in Ceph cluster-id instead
of a MON list from the storage class.

This helps in moving towards a stateless CSI implementation.

Tested the following,
- PV provisioning and staging using cluster-id in storage class
- PV provisioning and staging using MON list in storage class

Did not test,
- snapshot operations in either forms of the storage class

Signed-off-by: ShyamsundarR <srangana@redhat.com>
2019-03-26 16:19:24 +00:00
j-griffith
6ec1196f47 Rework multi-node-multi-writer feature
This commit reverts the initial implementation of the
multi-node-multi-writer feature:
  commit: b5b8e46460

It replaces that implementation with a more restrictive version that
only allows multi-node-multi-writer for volumes of type `block`

With this change there are no volume parameters required in the stoarge
class, we also fail any attempt to create a file based device with
multi-node-multi-write being specified, this way a user doesn't have to
wait until they try and do the publish before realizing it doesn't work.
2019-03-18 10:07:06 -06:00
j-griffith
a164169fd3 Revert "Add multiNodeWritable option for RBD Volumes"
This reverts commit b5b8e46460.
2019-03-13 18:26:46 -06:00
Madhu Rajanna
9f5ac5eeaa Fix return error codes
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-03-07 18:29:07 +05:30
Madhu Rajanna
16279eda78 Roundup volume size to Mib for rbd
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-03-04 19:17:28 +05:30
j-griffith
b5b8e46460 Add multiNodeWritable option for RBD Volumes
This change adds the ability to define a `multiNodeWritable` option in
the Storage Class.

This change does a number of things:
1. Allow multi-node-multi-writer access modes if the SC options is
enabled
2. Bypass the watcher checks for MultiNodeMultiWriter Volumes
3. Maintains existing watcher checks for SingleNodeWriter access modes
regardless of the StorageClass option.

fix lint-errors
2019-03-01 21:59:57 +00:00
Madhu Rajanna
e861f12a60 Fix review comments
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-02-26 14:35:16 +00:00
Madhu Rajanna
3b320ef19e Add support of RBD list volumes
currently all the created volumes are
stored in the metadata store, so we
can use this information to support
list volumes.

Signed-off-by: Madhu Rajanna <mrajanna@redhat.com>
2019-02-26 14:35:16 +00:00
Madhu Rajanna
f670c62526 Remove userID and key logging from rbd
Fixes: #171
2019-02-08 12:40:48 +05:30
Humble Chirammal
cb77ff5d87 Migrate rbd package to klog from glog
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2019-02-05 12:09:04 +00:00
Madhu Rajanna
5983fedddb Fix issues found in gometalinter static check
Signed-off-by: Madhu Rajanna <mrajanna@redhat.com>
2019-02-04 10:44:37 +05:30
Madhu Rajanna
50ba8ed446 Fix gometalinter issues
Signed-off-by: Madhu Rajanna <mrajanna@redhat.com>
2019-01-29 11:24:36 +05:30
Madhu Rajanna
ca2e475296 Fix gometalinter issues
Signed-off-by: Madhu Rajanna <mrajanna@redhat.com>
2019-01-29 11:23:50 +05:30
Madhu Rajanna
5eb1974e38 Fix vetshadow issues
Signed-off-by: Madhu Rajanna <mrajanna@redhat.com>
2019-01-25 14:15:25 +05:30
Madhu Rajanna
15b5b0112e rename Id to ID to fix lint issue
Signed-off-by: Madhu Rajanna <mrajanna@redhat.com>
2019-01-25 14:14:48 +05:30
Madhu Rajanna
9f76f6bd59 Remove dead code
Signed-off-by: Madhu Rajanna <mrajanna@redhat.com>
2019-01-25 14:10:52 +05:30
Madhu Rajanna
e42e66ff30 user errors.New if error formatting is not required
Signed-off-by: Madhu Rajanna <mrajanna@redhat.com>
2019-01-16 18:54:13 +05:30
Madhu Rajanna
20af5afcab Fix golint issues
Signed-off-by: Madhu Rajanna <mrajanna@redhat.com>
2019-01-16 18:33:38 +05:30
Huamin Chen
e46099a504 reconcile merge
Signed-off-by: Huamin Chen <hchen@redhat.com>
2019-01-15 16:20:41 +00:00
mickymiek
62d65ad0cb cm metadata persist for rbd and cephfs 2019-01-14 20:15:09 +00:00
Patrick Ohly
403cad682c rbd: protect against concurrent gRPC calls
The timeout value in external-provisioner is fairly low. It's not
uncommon that it times out and retries before the rbdplugin is done
with CreateVolume. rbdplugin has to serialize calls and ensure that
they are idempotent to deal with this.
2019-01-14 20:15:09 +00:00