Commit Graph

43 Commits

Author SHA1 Message Date
Prasanna Kumar Kalever
9a3170bf77 rbd: provide a way to disable the auto fallback to nbd mounter
This change allows the user to choose not to fallback to NBD mounter
when some ImageFeatures are absent with krbd driver, rather just fail
the NodeStage call.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-11-01 08:17:36 +00:00
Niels de Vos
b132696e54 rbd: note that thick-provisioning is deprecated
Thick-provisioning was introduced to make accounting of assigned space
for volumes easier. When thick-provisioned volumes are the only consumer
of the Ceph cluster, this works fine. However, it is unlikely that this
is the case. Instead, accounting of the requested (thin-provisioned)
size of volumes is much more practical as different types of volumes can
be tracked.

OpenShift already provides cluster-wide quotas, which can combine
accounting of requested volumes by grouping different StorageClasses.

In addition to the difficult practise of allowing only thick-provisioned
RBD backed volumes, the performance makes thick-provisioning
troublesome. As volumes need to be completely allocated, data needs to
be written to the volume. This can take a long time, depending on the
size of the volume. Provisioning, cloning and snapshotting becomes very
much noticeable, and because of the additional time consumption, more
prone to failures.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-10-27 06:54:07 +00:00
Prasanna Kumar Kalever
c9cc36d8db rbd: provide alternatives to preserve the ceph log files
Currently, we delete the ceph client log file on unmap/detach.

This patch provides additional alternatives for users who would like to
persist the log files.

Strategies:
-----------
`remove`: delete log file on unmap/detach
`compress`: compress the log file to gzip on unmap/detach
`preserve`: preserve the log file in text format

Note that the default strategy will be remove on unmap, and these options
can be tweaked from the storage class

Compression size details example:

On Map: (with debug-rbd=20)
---------
$ ls -lh
-rw-r--r-- 1 root root 526K Sep  1 18:15
rbd-nbd-0001-0024-fed5480a-f00f-417a-a51d-31d8a8144c03-0000000000000003-d2e89c87-0b4d-11ec-8ea6-160f128e682d.log

On unmap:
---------
$ ls -lh
-rw-r--r-- 1 root root  33K Sep  1 18:15
rbd-nbd-0001-0024-fed5480a-f00f-417a-a51d-31d8a8144c03-0000000000000003-d2e89c87-0b4d-11ec-8ea6-160f128e682d.gz

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-09-16 13:55:15 +00:00
Prasanna Kumar Kalever
314516cedd deploy: fix cephLogDir passing to storageclass via helm
cephLogDir: is a storage class option that is passed to rbd-nbd daemon.
cephLogDirHostPath: is a nodeplugin daemonset level option that helps in
                   using the right host-path while bind-mounting

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-09-16 13:55:15 +00:00
Prasanna Kumar Kalever
682b3a980b rbd: rbd-nbd logging the ceph-CSI way
- One logfile per device/volume
- Add ability to customize the logdir, default: /var/log/ceph

Note: if user customizes the hostpath to something else other than default
/var/log/ceph, then it is his responsibility to update the `cephLogDir`
in storageclass to reflect the same with daemon:

```
cephLogDir: "/var/log/mynewpath"
```

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-08-24 07:15:30 +00:00
Madhu Rajanna
d8f7b38d3d rbd: add exclusive-lock and journaling image features for rbd image
Current rbd plugin only supports the layering feature
for rbd image. Add exclusive-lock and journaling image
features for the rbd.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
Signed-off-by: woohhan <woohyung_han@tmax.co.kr>
2021-03-24 09:48:04 +00:00
Madhu Rajanna
c417a5d0ba rbd: add support for thick provisioning option
Add an option to the StorageClass to support creating fully allocated
(thick provisioned) RBD images

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-02-19 11:55:40 +00:00
Prasanna Kumar Kalever
d2f4666ebe deploy: group fields with optional/required labels
This patch with touch on the varuious other fields with in the storage class
yamls and label them with optional/required.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2020-10-07 13:01:45 +00:00
Prasanna Kumar Kalever
0c6d8fbd80 deploy: let the pool/fs name be required
Obviously expecting a pool with name `rbd` or CephFS name `myfs`
will be a limitation, as the pool/fs is created by admin manually,
let them choose the name that suits their requirement and come back
edit it in the storage class.

Making the pool/fs name as required field will give more attention,
else with new users it will be mostly left unedited until one hit
the errors saying no pool/fs exists.

This patch clips-off the default pool/fs name and make it a mandatory
field.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2020-10-07 13:01:45 +00:00
Madhu Rajanna
d1f175d9f3 rbd: add support for rbd map and unmap options
added support for providing map and unmap
options to rbd CLI when mapping rbd image
on the node.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-09-21 13:27:28 +00:00
Humble Chirammal
c1a269191c doc: make sure configmap object referred or documented correctly
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-06-10 11:33:10 +00:00
Yug Gupta
6cf7389d97 doc: Add prefix for naming subvolumes and snapshots
Updated storageclass and snapshotclass
to include the name prefix for naming
subvolumes and snapshots.

Fixes: #1087

Signed-off-by: Yug Gupta <ygupta@redhat.com>
2020-05-27 05:54:10 +00:00
ShyamsundarR
1a8f8e3c24 Add support for erasure coded pools
This commit adds support to mention dataPool parameter for the
topology constrained pools in the StorageClass, that can be
leveraged to mention erasure coded pool names to use for RBD
data instead of the replica pools.

Signed-off-by: ShyamsundarR <srangana@redhat.com>
2020-04-14 14:14:29 +00:00
ShyamsundarR
5c4abf8347 Add topology support to ceph-csi
Signed-off-by: ShyamsundarR <srangana@redhat.com>
2020-04-14 14:14:29 +00:00
Niels de Vos
825448825c examples: rbd/storageclass.yaml causes Python excetion with yamllint
While running the 'make test' target and have 'yamllint' available, the
test fails with the following exception:

yamllint -s -d {extends: default, rules: {line-length: {allow-non-breakable-inline-mappings: true}},ignore: charts/*/templates/*.yaml} ./examples/rbd/storageclass.yaml
Traceback (most recent call last):
  File "/usr/local/bin/yamllint", line 11, in <module>
    sys.exit(run())
  File "/usr/local/lib/python3.6/site-packages/yamllint/cli.py", line 181, in run
    problems = linter.run(f, conf, filepath)
  File "/usr/local/lib/python3.6/site-packages/yamllint/linter.py", line 237, in run
    content = input.read()
  File "/usr/lib64/python3.6/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 1947: ordinal not in range(128)

The quotes used in the comments seem to be non-ascii characters.
Replacing these with standard " makes the test pass again.

This problem occurred while running tests in a container based on the
Ceph image (CentOS-7) with Python 3. Travis CI might still use Python 2
for yamllint, and hide the problem.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-03-24 14:42:25 +00:00
Vasyl Purchel
669dc4536f Reduce encryption KMS configuration SC parameters
* moves KMS type from StorageClass into KMS configuration itself
 * updates omapval used to identify KMS to only it's ID without the type

why?

1. when using multiple KMS configurations (not currently supported)
automated parsing of kms configuration will be failing because some
entries in configs won't comply with the requested type
2. less options are needed in the StorageClass and less data used to
identify the KMS

Signed-off-by: Vasyl Purchel vasyl.purchel@workday.com
Signed-off-by: Andrea Baglioni andrea.baglioni@workday.com
2020-02-10 15:21:11 +00:00
Vasyl Purchel
419ad0dd8e Adds per volume encryption with Vault integration
- adds proposal document for PVC encryption from PR448
- adds per-volume encription by generating encryption passphrase
  for each volume and storing it in a KMS
- adds HashiCorp Vault integration as a KMS for encryption passphrases
- avoids encrypting volume second time if it was already encrypted but
  no file system created
- avoids unnecessary checks if volume is a mapped device when encryption
  was not requested
- prevents resizing encrypted volumes (it is not currently supported)
- prevents creating snapshots from encrypted volumes to prevent attack
  on encryption key (security guard until re-encryption of volumes
  implemented)

Signed-off-by: Vasyl Purchel vasyl.purchel@workday.com
Signed-off-by: Andrea Baglioni andrea.baglioni@workday.com

Fixes #420
Fixes #744
2020-02-05 05:18:56 +00:00
Vasyl Purchel
166eaf700f Adds PVC encryption with LUKS
Adds encryption in StorageClass as a parameter. Encryption passphrase is
stored in kubernetes secrets per StorageClass. Implements rbd volume
encryption relying on dm-crypt and cryptsetup using LUKS extension

The change is related to proposal made earlier. This is a first part of
the full feature that adds encryption with passphrase stored in secrets.

Signed-off-by: Vasyl Purchel vasyl.purchel@workday.com
Signed-off-by: Andrea Baglioni andrea.baglioni@workday.com
Signed-off-by: Ioannis Papaioannou ioannis.papaioannou@workday.com
Signed-off-by: Paul Mc Auley paul.mcauley@workday.com
Signed-off-by: Sergio de Carvalho sergio.carvalho@workday.com
2019-12-16 08:12:44 +00:00
Humble Chirammal
8e437244de Enable RBD resizer in travis
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2019-12-13 12:40:12 +00:00
Humble Chirammal
b44a81bdbc We were defaulting to ext4 at first and then moved toxfs.
However further testing shows that, ext4 should be
the default or preferred fs for RBD devices.

This patch bring that change

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2019-09-24 14:36:50 +05:30
Madhu Rajanna
6da96c6327 remove support for create image with image-format 1
tried to create an image with image-format=1

```
sh-4.2# rbd create --size=1024 replicapool/test --image-format=1
rbd: image format 1 is deprecated
rbd: create error: (22) Invalid argument
2019-09-11 07:00:54.531 7fb0e40bfb00 -1 librbd: Format 1 image creation unsupported.
```

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-09-12 07:52:32 +00:00
Madhu Rajanna
41b701c98c Add support for erasure pool in rbd
Allow specifying different metadata and data pools in a
CSI RBD StorageClass

Fixes: #199
Fixes: https://github.com/rook/rook/issues/2650
Fixes: https://github.com/rook/rook/issues/3763

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-09-11 06:48:08 +00:00
Madhu Rajanna
4ba2d0e10b Add xfs fstype as default type in storageclass
we have see better performace in device
format and mounting by setting the fstype to xfs
from default ext4.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-08-13 15:19:36 +00:00
Madhu Rajanna
f4c80dec9a Implement NodeStage and NodeUnstage for rbd
in NodeStage RPC call  we  have to map the
device to the node plugin and make  sure  the
the device will be mounted to  the global path

in  nodeUnstage request unmount the device from
global path and unmap the device

if the volume mode is block  we will be creating
a file inside a stageTargetPath  and it will be
considered  as the global path

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-07-24 12:49:21 +00:00
Madhu Rajanna
df2680c50b support mountflags during nodepublish in rbd
Fixes: #292

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-07-03 21:46:36 +05:30
ShyamsundarR
c5762b6b5c Modify RBD plugin to use a single ID and move the id and key into the secret
RBD plugin needs only a single ID to manage images and operations against a
pool, mentioned in the storage class. The current scheme of 2 IDs is hence not
needed and removed in this commit.

Further, unlike CephFS plugin, the RBD plugin splits the user id and the key
into the storage class and the secret respectively. Also the parameter name
for the key in the secret is noted in the storageclass making it a variant and
hampers usability/comprehension. This is also fixed by moving the id and the key
to the secret and not retaining the same in the storage class, like CephFS.

Fixes #270

Testing done:
- Basic PVC creation and mounting

Signed-off-by: ShyamsundarR <srangana@redhat.com>
2019-06-24 13:46:14 +00:00
ShyamsundarR
d02e50aa9b Removed config maps and replaced with rados omaps
Existing config maps are now replaced with rados omaps that help
store information regarding the requested volume names and the rbd
image names backing the same.

Further to detect cluster, pool and which image a volume ID refers
to, changes to volume ID encoding has been done as per provided
design specification in the stateless ceph-csi proposal.

Additional changes and updates,
- Updated documentation
- Updated manifests
- Updated Helm chart
- Addressed a few csi-test failures

Signed-off-by: ShyamsundarR <srangana@redhat.com>
2019-05-19 12:29:33 +00:00
Humble Chirammal
822cab8ed0 Make deployment artifacts with a consistent name pattern.
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2019-05-07 14:54:53 +00:00
ShyamsundarR
ba2e5cff51 Address remenant subject reference and code style reviews
Signed-off-by: ShyamsundarR <srangana@redhat.com>
2019-03-26 16:19:24 +00:00
ShyamsundarR
fc0cf957be Updated code and docs to reflect correct terminology
- Updated instances of fsid with clusterid
- Updated instances of credentials/subject with user/key

Signed-off-by: ShyamsundarR <srangana@redhat.com>
2019-03-26 16:19:24 +00:00
ShyamsundarR
2064e674a4 Addressed using k8s client APIs to fetch secrets
Based on the review comments addressed the following,
- Moved away from having to update the pod with volumes
when a new Ceph cluster is added for provisioning via the
CSI driver

- The above now used k8s APIs to fetch secrets
  - TBD: Need to add a watch mechanisim such that these
secrets can be cached and updated when changed

- Folded the Cephc configuration and ID/key config map
and secrets into a single secret

- Provided the ability to read the same config via mapped
or created files within the pod

Tests:
- Ran PV creation/deletion/attach/use using new scheme
StorageClass
- Ran PV creation/deletion/attach/use using older scheme
to ensure nothing is broken
- Did not execute snapshot related tests

Signed-off-by: ShyamsundarR <srangana@redhat.com>
2019-03-26 16:19:24 +00:00
ShyamsundarR
97f8c4b677 Provide options to pass in Ceph cluster-id
This commit provides the option to pass in Ceph cluster-id instead
of a MON list from the storage class.

This helps in moving towards a stateless CSI implementation.

Tested the following,
- PV provisioning and staging using cluster-id in storage class
- PV provisioning and staging using MON list in storage class

Did not test,
- snapshot operations in either forms of the storage class

Signed-off-by: ShyamsundarR <srangana@redhat.com>
2019-03-26 16:19:24 +00:00
John Griffith
d79ca5bc90 Merge branch 'csi-v1.0' into default_multiwrite_blockmode 2019-03-19 08:02:14 -06:00
j-griffith
a164169fd3 Revert "Add multiNodeWritable option for RBD Volumes"
This reverts commit b5b8e46460.
2019-03-13 18:26:46 -06:00
Madhu Rajanna
d61a87b42e Fix driver name as per CSI spec
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-03-13 12:04:30 +05:30
j-griffith
b5b8e46460 Add multiNodeWritable option for RBD Volumes
This change adds the ability to define a `multiNodeWritable` option in
the Storage Class.

This change does a number of things:
1. Allow multi-node-multi-writer access modes if the SC options is
enabled
2. Bypass the watcher checks for MultiNodeMultiWriter Volumes
3. Maintains existing watcher checks for SingleNodeWriter access modes
regardless of the StorageClass option.

fix lint-errors
2019-03-01 21:59:57 +00:00
Madhu Rajanna
cee9c4f8b2 Fix yamllint issues
Signed-off-by: Madhu Rajanna <mrajanna@redhat.com>
2019-02-07 12:19:14 +00:00
Huamin Chen
e4b24711f6 cope with latest changes in csi provisioner and deprecations 2019-01-23 10:58:50 -05:00
Huamin Chen
d5b7543565 allow monitors be embedded in credential secret
Signed-off-by: Huamin Chen <hchen@redhat.com>
2018-09-21 14:43:01 +00:00
Huamin Chen
4453cfce5b set dns policy in csi plugin so storage class can use mons' FQDN
Signed-off-by: Huamin Chen <hchen@redhat.com>
2018-09-19 14:39:43 +00:00
Huamin Chen
30a5d9a6e7 add rbd-nbd mounter in storage class
Signed-off-by: Huamin Chen <hchen@redhat.com>
2018-09-18 14:09:12 +00:00
Seungcheol Ko
b0e68a52e0 Refactoring using users 2018-08-09 22:07:00 +09:00
gman
e8ea0aa713 added examples 2018-08-07 15:11:22 +02:00