From cae8f529dc0d864c1677750320997e5553e45864 Mon Sep 17 00:00:00 2001 From: Mudit Agarwal Date: Thu, 14 May 2020 08:56:14 +0530 Subject: [PATCH] docs: add doc for cephFS static PV With the change in #382, support for static PV for CephFS was added. This change is to update the already existing doc for the same. Issue: #669 Signed-off-by: Mudit Agarwal --- docs/static-pvc.md | 133 ++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 131 insertions(+), 2 deletions(-) diff --git a/docs/static-pvc.md b/docs/static-pvc.md index 6971fafc3..8e1a210e3 100644 --- a/docs/static-pvc.md +++ b/docs/static-pvc.md @@ -6,8 +6,15 @@ - [Create RBD static PV](#create-rbd-static-pv) - [RBD Volume Attributes in PV](#rbd-volume-attributes-in-pv) - [Create RBD static PVC](#create-rbd-static-pvc) + - [CephFS static PVC](#cephfs-static-pvc) + - [Create CephFS subvolume](#create-cephfs-subvolume) + - [Create CephFS static PV](#create-cephfs-static-pv) + - [Node stage secret ref in CephFS PV](#node-stage-secret-ref-in-cephfs-pv) + - [CephFS Volume Attributes in PV](#cephfs-volume-attributes-in-pv) + - [Create CephFS static PVC](#create-cephfs-static-pvc) -This document outlines how to create static PV and static PVC from existing rbd image +This document outlines how to create static PV and static PVC from +existing rbd image/cephFS volume. **warning** static PVC can be created, deleted, mounted and unmounted but currently ceph-csi doesn't support other operations like snapshot,clone, @@ -26,7 +33,7 @@ to access by the application pod you can skip this step. Lets create a new rbd image in ceph cluster which we are going to use for static PVC -```bash +```console [$]rbd create static-image --size=1024 --pool=replicapool ``` @@ -117,3 +124,125 @@ persistentvolumeclaim/fs-static-pvc created **Note** deleting PV and PVC doesnot deleted the backend rbd image, user need to manually delete the rbd image if required + +## CephFS static PVC + +CephFS subvolume created manually can be mounted and unmounted to an app, +below steps show how to create a CephFS subvolume, static PV and static PVC. + +### Create CephFS subvolume + +If you already have a CephFS subvolume created and contains some data +which you want to access by the application pod you can skip this step. + +Lets create a new CephFS subvolume of size 1 GiB in ceph cluster which +we are going to use for static PVC, before that we need to create +the subvolumegroup. **myfs** is the filesystem name(volume name) inside +which subvolume should be created. + +```console +[$]ceph fs subvolumegroup create myfs testGroup +``` + +```console +[$]ceph fs subvolume create myfs testSubVolume testGroup --size=1073741824 +``` + +**Note:** volume here refers to the filesystem. + +### Create CephFS static PV + +To create the CephFS PV you need to know the `volume rootpath`, and `clusterID`, +here is the command to get the root path in ceph cluster + +```bash +[$]ceph fs subvolume getpath myfs testSubVolume testGroup +/volumes/testGroup/testSubVolume +``` + +```yaml +apiVersion: v1 +kind: PersistentVolume +metadata: + name: cephfs-static-pv +spec: + accessModes: + - ReadWriteMany + capacity: + storage: 1Gi + csi: + driver: cephfs.csi.ceph.com + nodeStageSecretRef: + # node stage secret name + name: csi-cephfs-secret + # node stage secret namespace where above secret is created + namespace: default + volumeAttributes: + # Required options from storageclass parameters need to be added in volumeAttributes + "clusterID": "ba68226a-672f-4ba5-97bc-22840318b2ec" + "fsName": "myfs" + "staticVolume": "true" + "rootPath": /volumes/testGroup/testSubVolume + # volumeHandle can be anything, need not to be same + # as PV name or volume name. keeping same for brevity + volumeHandle: cephfs-static-pv + persistentVolumeReclaimPolicy: Retain + volumeMode: Filesystem +``` + +### Node stage secret ref in CephFS PV + +For static CephFS PV to work, userID and userKey needs to be specified in the +secret. Static PV will not work with adminID and adminKey. +Format for the secret should be same as detailed [here](https://github.com/ceph/ceph-csi/blob/3e656769b71a3c43d95f6875ed4934c82a8046e7/examples/cephfs/secret.yaml#L7-L10). + +### CephFS volume attributes in PV + +Below table explains the list of volume attributes can be set when creating a +static CephFS PV + +| Attributes | Description | Required | +| :----------: | :----------------------------------------------------------------------------------------------------------------------------------------: | :------: | +| clusterID | The is used by the CSI plugin to uniquely identify and use a Ceph cluster (this is the key in configmap created duing ceph-csi deployment) | Yes | +| fsName | CephFS filesystem name into which the subvolume should be created/present | Yes | +| staticVolume | Value must be set to `true` to mount and unmount static rbd PVC | Yes | +| rootPath | Actual path of the subvolume in ceph cluster, can be retrieved by issuing getpath command as described above | Yes | + +**Note** ceph-csi does not supports CephFS subvolume deletion for static PV. +`persistentVolumeReclaimPolicy` in PV spec must be set to `Retain` to avoid PV +delete attempt in csi-provisioner. + +```bash +[$] kubectl create -f cephfs-static-pv.yaml +persistentvolume/cephfs-static-pv created +``` + +### Create CephFS static PVC + +To create the CephFS PVC you need to know the PV name which is created above + +```yaml +apiVersion: v1 +kind: PersistentVolumeClaim +metadata: + name: cephfs-static-pvc + namespace: default +spec: + accessModes: + - ReadWriteMany + resources: + requests: + storage: 1Gi + volumeMode: Filesystem + # volumeName should be same as PV name + volumeName: cephfs-static-pv +``` + +```bash +[$] kubectl create -f cephfs-static-pvc.yaml +persistentvolumeclaim/cephfs-static-pvc created +``` + +**Note** deleting PV and PVC doesnot deleted the backend CephFS subvolume, +user needs to manually delete the CephFS subvolume if required +