1
0
mirror of https://github.com/ceph/ceph-csi.git synced 2024-12-25 22:40:21 +00:00
Commit Graph

162 Commits

Author SHA1 Message Date
Madhu Rajanna
96a3aabe5a deploy: remove psp from cephcsi
as PSP is deprecated in kubernetes 1.21
and will be removed in kubernetes 1.25
removing the existing PSP related templates
from the repo and updated the required documents.

fixes 

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-08-23 07:53:46 +00:00
Rakshith R
0c33a33d5c rbd: add kmip encryption type
The Key Management Interoperability Protocol (KMIP)
is an extensible communication protocol
that defines message formats for the manipulation
of cryptographic keys on a key management server.
Ceph-CSI can now be configured to connect to
various KMS using KMIP for encrypting RBD volumes.

https://en.wikipedia.org/wiki/Key_Management_Interoperability_Protocol

Signed-off-by: Rakshith R <rar@redhat.com>
2022-08-18 07:41:42 +00:00
Rakshith R
48d66d6cfd deploy: modify nfs daemonset to use cephcsi nfs nodeserver
This commit makes modification to nfs daemonset to use
nfs nodeserver. `nfs.NetNamespaceFilePath` example is
added.

Signed-off-by: Rakshith R <rar@redhat.com>
2022-08-09 13:36:03 +00:00
Robert Vasek
f59806caff doc: added docs for snapshot-backed CephFS volumes
Signed-off-by: Robert Vasek <robert.vasek@cern.ch>
2022-06-16 09:44:27 +00:00
Madhu Rajanna
4b57cc3ec5 rbd: add support for rbd striping
RBD supports creating rbd images with
object size, stripe unit and stripe count
to support striping. This PR adds the support
for the same.

More details about striping at
https://docs.ceph.com/en/quincy/man/8/rbd/#striping

fixes: 

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-06-09 18:59:00 +00:00
Rakshith R
0ddfdc5b69 deploy: add nfs pvc-clone & pod-clone example yaml
Signed-off-by: Rakshith R <rar@redhat.com>
2022-05-24 18:13:02 +00:00
Rakshith R
5983d5730a deploy: add support for nfs snapshot
This commit add external-snapshotter container and
required rbac to support nfs snaphots.
Example volumesnaphotclass, volumesnapshot,
pvc-restore and pod-restore yamls are also added.

Signed-off-by: Rakshith R <rar@redhat.com>
2022-05-24 18:13:02 +00:00
Niels de Vos
891a840aa6 deploy: add resize support to NFS-provisioner deployment
Resizing is handled by the csi-resizer container, which needs to run in
the provisioner Pod. In addition to the container, the StorageClass also
needs to allow volume expansion.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-05-10 17:43:59 +00:00
Niels de Vos
085a3bdc73 doc: use .nfs as default pool for NFS-export configs
"nfs-ganesha" is the default pool for older Ceph versions, recent
versions use ".nfs" (which can not be changed in the CephNFS resource).

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-05-10 00:43:43 +00:00
Niels de Vos
c137af57f3 e2e: add RWOP examples for NFS-provisioning
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-05-10 00:43:43 +00:00
Madhu Rajanna
d2bc9743f7 cephfs: add netNamespaceFilePath for CephFS
as same host directory is not shared between
the cephfs and the rbd plugin pod. we need
to keep the netNamespaceFilePath separately
for both cephfs and rbd. CephFS plugin will
use this path to execute mount -t commands.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-04-19 12:28:46 +00:00
Madhu Rajanna
b4acbd08a5 rbd: move radosNamespace to RBD section
As radosNamespace is more specific to
RBD not the general ceph configuration. Now
we introduced a new RBD section for RBD specific
options, Moving the radosNamespace to RBD section
and keeping the radosNamespace still under the
global ceph level configration for backward
compatibility.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-04-19 12:28:46 +00:00
Madhu Rajanna
766346868e util: Add RBD specific options in clusterInfo
As the netNamespaceFilePath can be separate for
both cephfs and rbd adding the netNamespaceFilePath
path for RBD, This will help us to keep RBD and
CephFS specific options separately.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-04-19 12:28:46 +00:00
Madhu Rajanna
7b2aef0d81 util: add support for the nsenter
add support to run rbd map and mount -t
commands with the nsenter.

complete design of pod/multus network
is added here https://github.com/rook/rook/
blob/master/design/ceph/multus-network.md#csi-pods

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-04-08 10:23:21 +00:00
Niels de Vos
190504713a doc: initial/partial instructions for using NFS examples
The README explains some of the requirements and basic configuration for
using the NFS-provisioner. When more deployment artifacts are added, the
README will get extended.

The Rook CephNFS example is included, as it is the easiest to get
started with dynamic provisioning of NFS-volumes.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-03-28 11:58:42 +00:00
Niels de Vos
b72774f9e9 doc: example for PVC and Pod using a NFS-volume
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-03-28 11:58:42 +00:00
Niels de Vos
2743510009 doc: example of StorageClass for NFS-provisioning
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-03-28 11:58:42 +00:00
Rakshith R
23145bc1f7 cleanup: fix csi-kms-connection-details.yaml ibmkeyprotect-test ex
The last option in `ibmkeyprotect-test` example had a extra `,`
at the end, which caused parsing to fail. This commit removes
the same.

Signed-off-by: Rakshith R <rar@redhat.com>
2022-03-16 07:29:56 +00:00
Rakshith R
4f0bb2315b rbd: add aws-sts-metdata encryption type
With Amazon STS and kubernetes cluster is configured with
OIDC identity provider, credentials to access Amazon KMS
can be fetched using oidc-token(serviceaccount token).
Each tenant/namespace needs to create a secret with aws region,
role and CMK ARN.
Ceph-CSI will assume the given role with oidc token and access
aws KMS, with given CMK to encrypt/decrypt DEK which will stored
in the image metdata.

Refer: https://docs.aws.amazon.com/STS/latest/APIReference/welcome.html
Resolves: 

Signed-off-by: Rakshith R <rar@redhat.com>
2022-03-16 07:29:56 +00:00
Rakshith R
dec74a534f doc: add step to create ceph-conf cm to deploy plugin
Ceph-csi requires `ceph-config` configmap when deploying.
Add this step to `examples/README.md`.

Fixes: 

Signed-off-by: Rakshith R <rar@redhat.com>
2022-03-09 06:39:57 +00:00
Madhu Rajanna
dd96024620 doc: remove mention of image feature dependency
avoid specifying the image feature dependencies
and add a link to rbd official document for
reference to the image feature dependencies.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-03-02 09:08:33 +00:00
Madhu Rajanna
d5c98f81a2 rbd: make image features as optional parameter
Makes the rbd images features in the storageclass
as optional so that default image features of librbd
can be used. and also kept the option to user
to specify the image features in the storageclass.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-02-28 13:10:03 +00:00
Madhu Rajanna
fb3835691f rbd: add support for deep-flatten image feature
as deep-flatten is long supported in ceph and its
enabled by default in the librbd, providing an option
to enable it in cephcsi for the rbd images we are
creating.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-02-28 13:10:03 +00:00
Rakshith R
3203673d17 cleanup: remove ceph.conf WA options which are already fixed
This commit removes ceph.conf WA options:
```
     # Workaround for http://tracker.ceph.com/issues/23446
     fuse_set_user_groups = false

     # ceph-fuse which uses libfuse2 by default has write buffer size of 2KiB
     # adding 'fuse_big_writes = true' option by default to override this limit
     # see https://github.com/ceph/ceph-csi/issues/1928
     fuse_big_writes = true
```
Since they are already fixed.

Refer: https://tracker.ceph.com/issues/44885
Refer: https://tracker.ceph.com/issues/23446
Closes: 

Signed-off-by: Rakshith R <rar@redhat.com>
2022-02-04 15:42:32 +00:00
Humble Chirammal
13f1611396 deploy: correct typos in aws encryption yamls
The field name was wrong in example yaml and this correct the same

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-02-02 14:26:27 +00:00
Madhu Rajanna
28fef9b379 cleanup: remove thick provisioning code
This commit removes the thick provisioning
code as thick provisioning is deprecated in
cephcsi 3.5.0.

fixes: 

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-01-28 11:17:15 +00:00
Humble Chirammal
f822600689 rbd: change the keyprotect metadata name to ibmkeyprotect
To be consistent with other components and also to explictly
state it belong to `ibm keyprotect` service introducing this
change

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-01-26 02:28:05 +00:00
Humble Chirammal
8f7016229a e2e: add example yamls for RWOP POD and PVC
This commit add example yamls for RBD and CephFS Pod and PVCs

RBD:
 Raw Block Volume and File Mode PVCs with RWOP accessmode
 Raw Block Volume POD and FileMode POD yamls referring RWOP PVC

CephFS:
 RWOP PVC and POD yaml referring RWOP PVC

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-01-11 19:40:22 +00:00
Humble Chirammal
bc354b6fb5 rbd: add BaseURL and tokenURL configuration
This commit adds optional BaseURL and TokenURL configuration to
key protect/hpcs configuration and client connections, if not
provided default values are used.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-01-11 21:12:56 +05:30
Humble Chirammal
5aa1e4d225 rbd: change the configmap of HPCS/KP key names to reflect the IBM string
considering IBM has different crypto services (ex: SKLM) in place, its
good to keep the configmap key names with below format

`IBM_KP_...` instead of `KP_..`

so that in future, if we add more crypto services from IBM we can keep
similar schema specific to that specific service from IBM.

Ex: `IBM_SKLM_...`

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-01-05 06:08:19 +00:00
Madhu Rajanna
494c9b91cb e2e: add e2e for bigger size restore
added e2e for below cases

Normal PVC snapshot restore to a bigger
size PVC (without encryption)

* Filesystem pvc restore to a bigger size
* Block pvc restore to a bigger size

Encrypted PVC snapshot restore to a bigger
size PVC

* Filesystem pvc restore to a bigger size
* Block pvc restore to a bigger size

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-12-23 03:47:00 +00:00
Humble Chirammal
9200bc7a00 rbd: Implement Key Protect KMS integration for Ceph CSI
This commit adds the support for HPCS/Key Protect IBM KMS service
to Ceph CSI service. EncryptDEK() and DecryptDEK() of RBD volumes are
done with the help of key protect KMS server by wrapping and unwrapping
the DEK and by using the DEKStoreMetadata.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-12-21 17:09:50 +00:00
Konstantin Shalygin
7411773f73 rbd: added RBD features support for krbd
Added support for `object-map, fast-diff`

Signed-off-by: Konstantin Shalygin <k0ste@k0ste.ru>
2021-12-07 07:38:24 +00:00
Prasanna Kumar Kalever
ec8397d8d9 deploy: updated sc templates for map/unmap Options with the new schema
Fixes: 
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-11-23 08:54:37 +00:00
Humble Chirammal
e6949945bb cephfs: add validation for generic ephemeral volumes
This commit adds the validation of csi cephfs driver to work with
ephemeral volume support. With ephemeral volume support a user can
specify ephemeral volumes in its pod spec and tie the lifecycle
of the PVC with the POD.

An example POD spec also included in this commit.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-11-22 06:08:23 +00:00
Humble Chirammal
df0901ddd8 rbd: add generic ephemeral volume validation
This commit adds the validation of csi RBD driver to work with
ephemeral volume support. With ephemeral volume support a user can
specify ephemeral volumes in its pod spec and tie the lifecycle
of the PVC with the POD.

An example pod spec is also included in this commit.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-11-22 06:08:23 +00:00
Madhu Rajanna
5524b2d538 ci: use 1.8.5 vault for e2e
current latest vault release is 1.9.0 but
with the latest image our E2E is broken.
reverting back the vault version to 1.8.5
till we root cause the issue.

Note:- This is to unblock PR merging

updates: 

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-11-19 10:37:14 +00:00
Prasanna Kumar Kalever
9a3170bf77 rbd: provide a way to disable the auto fallback to nbd mounter
This change allows the user to choose not to fallback to NBD mounter
when some ImageFeatures are absent with krbd driver, rather just fail
the NodeStage call.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-11-01 08:17:36 +00:00
Niels de Vos
b132696e54 rbd: note that thick-provisioning is deprecated
Thick-provisioning was introduced to make accounting of assigned space
for volumes easier. When thick-provisioned volumes are the only consumer
of the Ceph cluster, this works fine. However, it is unlikely that this
is the case. Instead, accounting of the requested (thin-provisioned)
size of volumes is much more practical as different types of volumes can
be tracked.

OpenShift already provides cluster-wide quotas, which can combine
accounting of requested volumes by grouping different StorageClasses.

In addition to the difficult practise of allowing only thick-provisioned
RBD backed volumes, the performance makes thick-provisioning
troublesome. As volumes need to be completely allocated, data needs to
be written to the volume. This can take a long time, depending on the
size of the volume. Provisioning, cloning and snapshotting becomes very
much noticeable, and because of the additional time consumption, more
prone to failures.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-10-27 06:54:07 +00:00
Niels de Vos
85c84910d3 e2e: add a monitor container to the vault Pod
The command `vault monitor` can be used to stream logging from the Vault
service. This is very helpful while debugging Vault configuration
failures.

By adding a 2nd container to the Vault deployment, it is now possible to
get the messages from the Vault service by running

    $ kubectl logs -c monitor <vault-pod-0123abcd>

This will be very useful when the e2e tests do not delete the deployment
after a failure and fetch the logs from all containers.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-10-19 03:37:42 +00:00
Yati Padia
1cf14cd83c cleanup: rework on naming conventions
This commits replaces cephfs -> cephFS
to maintain consistency throughout the
codebase

Updates: 

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-09-24 06:17:17 +00:00
Prasanna Kumar Kalever
c9cc36d8db rbd: provide alternatives to preserve the ceph log files
Currently, we delete the ceph client log file on unmap/detach.

This patch provides additional alternatives for users who would like to
persist the log files.

Strategies:
-----------
`remove`: delete log file on unmap/detach
`compress`: compress the log file to gzip on unmap/detach
`preserve`: preserve the log file in text format

Note that the default strategy will be remove on unmap, and these options
can be tweaked from the storage class

Compression size details example:

On Map: (with debug-rbd=20)
---------
$ ls -lh
-rw-r--r-- 1 root root 526K Sep  1 18:15
rbd-nbd-0001-0024-fed5480a-f00f-417a-a51d-31d8a8144c03-0000000000000003-d2e89c87-0b4d-11ec-8ea6-160f128e682d.log

On unmap:
---------
$ ls -lh
-rw-r--r-- 1 root root  33K Sep  1 18:15
rbd-nbd-0001-0024-fed5480a-f00f-417a-a51d-31d8a8144c03-0000000000000003-d2e89c87-0b4d-11ec-8ea6-160f128e682d.gz

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-09-16 13:55:15 +00:00
Prasanna Kumar Kalever
314516cedd deploy: fix cephLogDir passing to storageclass via helm
cephLogDir: is a storage class option that is passed to rbd-nbd daemon.
cephLogDirHostPath: is a nodeplugin daemonset level option that helps in
                   using the right host-path while bind-mounting

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-09-16 13:55:15 +00:00
Madhu Rajanna
4dba4e3f6f deploy: add configmap for ceph.conf
added an example configmap for the
ceph.conf file. This allows the users to
specify different ceph configuration when
deploying cephcsi.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-09-03 14:14:43 +00:00
Prasanna Kumar Kalever
682b3a980b rbd: rbd-nbd logging the ceph-CSI way
- One logfile per device/volume
- Add ability to customize the logdir, default: /var/log/ceph

Note: if user customizes the hostpath to something else other than default
/var/log/ceph, then it is his responsibility to update the `cephLogDir`
in storageclass to reflect the same with daemon:

```
cephLogDir: "/var/log/mynewpath"
```

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-08-24 07:15:30 +00:00
Niels de Vos
bb60173a98 e2e: add verifyKeyDestroyed() for validating vaultDestroyKeys
The kmsConfig type in the e2e suite has been enhanced with two functions
that make it possible to validate the destruction of deleted keys.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-08-06 12:19:18 +00:00
Niels de Vos
f584db41e6 util: add vaultDestroyKeys option to destroy Vault kv-v2 secrets
Hashicorp Vault does not completely remove the secrets in a kv-v2
backend when the keys are deleted. The metadata of the keys will be
kept, and it is possible to recover the contents of the keys afterwards.

With the new `vaultDestroyKeys` configuration parameter, this behaviour
can now be selected. By default the parameter will be set to `true`,
indicating that the keys and contents should completely be destroyed.
Setting it to any other value will make it possible to recover the
deleted keys.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-08-06 12:19:18 +00:00
Madhu Rajanna
d321663872 deploy: add template changes for mapping
added template changes for the clusterID and
poolID,fsID mapping details for the pod templates.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-08-05 16:07:51 +00:00
Niels de Vos
f2d5c2e0df util: add vaultAuthNamespace option for Vault KMS
The new `vaultAuthNamespace` configuration parameter can be set to the
Vault Namespace where the authentication is setup in the service. Some
Hashicorp Vault deployments use sub-namespaces for their users/tenants,
with a 'root' namespace where the authentication is configured. This
requires passing of different Vault namespaces for different operations.

Example:
 - the Kubernetes Auth mechanism is configured for in the Vault
   Namespace called 'devops'
 - a user/tenant has a sub-namespace called 'devops/website' where the
   encryption passphrases can be placed in the key-value store

The configuration for this, then looks like:

    vaultAuthNamespace: devops
    vaultNamespace: devops/homepage

Note that Vault Namespaces are a feature of the Hashicorp Vault
Enterprise product, and not part of the Open Source version. This
prevents adding e2e tests that validate the Vault Namespace
configuration.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-08-04 18:20:45 +00:00
Niels de Vos
e0ac70f8fb e2e: use official CentOS container location
registry.centos.org is not officially maintained by the CentOS
infrastructure team. The container images on quay.io are the official
once and we should use those instead.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-08-03 12:19:46 +00:00