mirrors/ceph-csi
1
0
Fork 0
mirror of https://github.com/ceph/ceph-csi.git synced 2024-12-18 02:50:30 +00:00
Code Issues Packages Projects Releases Wiki Activity

doc: guide for static PVC creation

Browse Source 
Add a detailed guide on creating a static PVC for a CephFS volume

Signed-off-by: Yilong Li <45418105+MondayCha@users.noreply.github.com>
This commit is contained in:
Yilong Li 2024-12-03 20:33:07 +08:00 committed by mergify[bot]
parent 8f4520ece5
commit 77d8306504
1 changed files with 126 additions and 27 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

153
docs/static-pvc.md
View File

@ -7,31 +7,35 @@
- [RBD Volume Attributes in PV](#rbd-volume-attributes-in-pv) - [RBD Volume Attributes in PV](#rbd-volume-attributes-in-pv)
- [Create RBD static PVC](#create-rbd-static-pvc) - [Create RBD static PVC](#create-rbd-static-pvc)
- [Resize RBD image](#resize-rbd-image) - [Resize RBD image](#resize-rbd-image)
- [Verify RBD static PVC](#verify-rbd-static-pvc)
- [CephFS static PVC](#cephfs-static-pvc) - [CephFS static PVC](#cephfs-static-pvc)
- [Create CephFS subvolume](#create-cephfs-subvolume) - [Create CephFS subvolume](#create-cephfs-subvolume)
- [Create CephFS static PV](#create-cephfs-static-pv) - [Create CephFS static PV](#create-cephfs-static-pv)
- [Node stage secret ref in CephFS PV](#node-stage-secret-ref-in-cephfs-pv) - [Node stage secret ref in CephFS PV](#node-stage-secret-ref-in-cephfs-pv)
- [CephFS volume attributes in PV](#cephfs-volume-attributes-in-pv) - [CephFS volume attributes in PV](#cephfs-volume-attributes-in-pv)
- [Create CephFS static PVC](#create-cephfs-static-pvc) - [Create CephFS static PVC](#create-cephfs-static-pvc)
- [Verify CephFS static PVC](#verify-cephfs-static-pvc)
This document outlines how to create static PV and static PVC from This document outlines how to create static PV and static PVC from
existing rbd image/cephFS volume. existing RBD image or CephFS volume.
**warning** static PVC can be created, deleted, mounted and unmounted but > [!warning]
> static PVC can be created, deleted, mounted and unmounted but
currently ceph-csi doesn't support other operations like snapshot,clone, currently ceph-csi doesn't support other operations like snapshot,clone,
resize, etc for static PVC resize, etc for static PVC
## RBD static PVC ## RBD static PVC
RBD images created manually can be mounted and unmounted to an app, below step RBD images created manually can be mounted and unmounted to an app, below step
shows how to create a rbd image, static PV, static PVC shows how to create a RBD image, static PV, static PVC
### Create RBD image ### Create RBD image
If you already have a rbd image created and contains some data which you want > [!tip]
> If you already have a RBD image created and contains some data which you want
to access by the application pod you can skip this step. to access by the application pod you can skip this step.
Lets create a new rbd image in ceph cluster which we are going to use for Let's create a new RBD image in ceph cluster which we are going to use for
static PVC static PVC
```console ```console
@ -40,8 +44,8 @@ rbd create static-image --size=1024 --pool=replicapool
### Create RBD static PV ### Create RBD static PV
To create the rbd PV you need to know the `rbd image name`,`clusterID` and To create the RBD PV you need to know the `rbd image name`,`clusterID` and
`pool` name in which the rbd image is created `pool` name in which the RBD image is created
```yaml ```yaml
apiVersion: v1 apiVersion: v1
@ -83,12 +87,13 @@ static RBD PV
| Attributes | Description | Required | | Attributes | Description | Required |
| :-----------: | :--------------------------------------------------------------------------------------------------------------------------------------------------: | :------: | | :-----------: | :--------------------------------------------------------------------------------------------------------------------------------------------------: | :------: |
| clusterID | The clusterID is used by the CSI plugin to uniquely identify and use a Ceph cluster (this is the key in configmap created duing ceph-csi deployment) | Yes | | clusterID | The clusterID is used by the CSI plugin to uniquely identify and use a Ceph cluster (this is the key in configmap created duing ceph-csi deployment) | Yes |
| pool | The pool name in which rbd image is created | Yes | | pool | The pool name in which RBD image is created | Yes |
| staticVolume | Value must be set to `true` to mount and unmount static rbd PVC | Yes | | staticVolume | Value must be set to `true` to mount and unmount static RBD PVC | Yes |
| imageFeatures | CSI RBD currently supports `layering, journaling, exclusive-lock` features. If `journaling` is enabled, must enable `exclusive-lock` too | Yes | | imageFeatures | CSI RBD currently supports `layering, journaling, exclusive-lock` features. If `journaling` is enabled, must enable `exclusive-lock` too | Yes |
| mounter | If set to `rbd-nbd`, use `rbd-nbd` on nodes that have `rbd-nbd` and `nbd` kernel modules to map rbd images | No | | mounter | If set to `rbd-nbd`, use `rbd-nbd` on nodes that have `rbd-nbd` and `nbd` kernel modules to map RBD images | No |
**Note** ceph-csi does not supports rbd image deletion for static PV. > [!note]
> ceph-csi does not supports RBD image deletion for static PV.
`persistentVolumeReclaimPolicy` in PV spec must be set to `Retain` to avoid PV `persistentVolumeReclaimPolicy` in PV spec must be set to `Retain` to avoid PV
delete attempt in csi-provisioner. delete attempt in csi-provisioner.
@ -99,7 +104,7 @@ persistentvolume/fs-static-pv created
### Create RBD static PVC ### Create RBD static PVC
To create the rbd PVC you need to know the PV name which is created above To create the RBD PVC you need to know the PV name which is created above
```yaml ```yaml
apiVersion: v1 apiVersion: v1
@ -134,32 +139,78 @@ Let us resize the RBD image in ceph cluster
rbd resize static-image --size=2048 --pool=replicapool rbd resize static-image --size=2048 --pool=replicapool
``` ```
Once the rbd image is resized in the ceph cluster, update the PV size and PVC Once the RBD image is resized in the ceph cluster, update the PV size and PVC
size to match the size of the rbd image. size to match the size of the RBD image.
Now scale down the application pod which is using `cephfs-static-pvc` and scale Now scale down the application pod which is using `cephfs-static-pvc` and scale
up the application pod to resize the filesystem. up the application pod to resize the filesystem.
**Note** If you have mounted same static PVC to multiple application pods, make > [!note]
> If you have mounted same static PVC to multiple application pods, make
sure you will scale down all the application pods and make sure no application sure you will scale down all the application pods and make sure no application
pods using the static PVC is running on the node and scale up all the pods using the static PVC is running on the node and scale up all the
application pods again(this will trigger `NodeStageVolumeRequest` which will application pods again(this will trigger `NodeStageVolumeRequest` which will
resize the filesystem for static volume). resize the filesystem for static volume).
**Note** deleting PV and PVC does not removed the backend rbd image, user need to ### Verify RBD static PVC
manually delete the rbd image if required
We can verify the RBD static PVC by creating a Pod that mounts the PVC.
```yaml
apiVersion: v1
kind: Pod
metadata:
name: rbd-static-pvc-test
namespace: default
spec:
containers:
- name: busybox
image: busybox:latest
volumeMounts:
- name: static-pvc
mountPath: /data/pvc
command: ["sleep", "3600"]
volumes:
- name: static-pvc
persistentVolumeClaim:
claimName: fs-static-pvc
```
```bash
$ kubectl create -f rbd-static-pvc-test.yaml
pod/rbd-static-pvc-test created
```
Verify that the PVC has been successfully mounted.
```bash
kubectl exec rbd-static-pvc-test -- df -h /data/pvc
```
Once you have completed the verification step, you can delete the test Pod for RBD.
```bash
$ kubectl delete pod rbd-static-pvc-test
pod "rbd-static-pvc-test" deleted
```
> [!note]
> deleting PV and PVC does not removed the backend RBD image, user need to
manually delete the RBD image if required
## CephFS static PVC ## CephFS static PVC
CephFS subvolume created manually can be mounted and unmounted to an app, CephFS subvolume or volume created manually can be mounted and unmounted
below steps show how to create a CephFS subvolume, static PV and static PVC. to an app, below steps show how to create a CephFS subvolume or volume,
static PV and static PVC.
### Create CephFS subvolume ### Create CephFS subvolume
If you already have a CephFS subvolume created and contains some data > [!tip]
which you want to access by the application pod you can skip this step. > If you already have a CephFS subvolume or volume created and contains some data
which you want to access by the application pod, you can skip this step.
Lets create a new CephFS subvolume of size 1 GiB in ceph cluster which Let's create a new CephFS subvolume of size 1 GiB in ceph cluster which
we are going to use for static PVC, before that we need to create we are going to use for static PVC, before that we need to create
the subvolumegroup. **myfs** is the filesystem name(volume name) inside the subvolumegroup. **myfs** is the filesystem name(volume name) inside
which subvolume should be created. which subvolume should be created.
@ -172,18 +223,22 @@ ceph fs subvolumegroup create myfs testGroup
ceph fs subvolume create myfs testSubVolume testGroup --size=1073741824 ceph fs subvolume create myfs testSubVolume testGroup --size=1073741824
``` ```
**Note:** volume here refers to the filesystem. > [!note]
> volume here refers to the filesystem.
### Create CephFS static PV ### Create CephFS static PV
To create the CephFS PV you need to know the `volume rootpath`, and `clusterID`, To create the CephFS PV you need to know the `volume rootpath`, and `clusterID`,
here is the command to get the root path in ceph cluster here is the command to get the root path of subvolume in ceph cluster
```bash ```bash
$ ceph fs subvolume getpath myfs testSubVolume testGroup $ ceph fs subvolume getpath myfs testSubVolume testGroup
/volumes/testGroup/testSubVolume /volumes/testGroup/testSubVolume
``` ```
For volume, you can directly use the folder path relative to the
filesystem as the `rootpath`.
```yaml ```yaml
apiVersion: v1 apiVersion: v1
kind: PersistentVolume kind: PersistentVolume
@ -231,7 +286,7 @@ static CephFS PV
| clusterID | The clusterID is used by the CSI plugin to uniquely identify and use a Ceph cluster (this is the key in configmap created duing ceph-csi deployment) | Yes | | clusterID | The clusterID is used by the CSI plugin to uniquely identify and use a Ceph cluster (this is the key in configmap created duing ceph-csi deployment) | Yes |
| fsName | CephFS filesystem name to be mounted. Not passing this option mounts the default file system. | No | | fsName | CephFS filesystem name to be mounted. Not passing this option mounts the default file system. | No |
| staticVolume | Value must be set to `true` to mount and unmount static cephFS PVC | Yes | | staticVolume | Value must be set to `true` to mount and unmount static cephFS PVC | Yes |
| rootPath | Actual path of the subvolume in ceph cluster, can be retrieved by issuing getpath command as described above | Yes | | rootPath | Actual path of the subvolume in ceph cluster which can be retrieved by issuing getpath command as described above, or folder path of the volume | Yes |
**Note** ceph-csi does not supports CephFS subvolume deletion for static PV. **Note** ceph-csi does not supports CephFS subvolume deletion for static PV.
`persistentVolumeReclaimPolicy` in PV spec must be set to `Retain` to avoid PV `persistentVolumeReclaimPolicy` in PV spec must be set to `Retain` to avoid PV
@ -269,5 +324,49 @@ $ kubectl create -f cephfs-static-pvc.yaml
persistentvolumeclaim/cephfs-static-pvc created persistentvolumeclaim/cephfs-static-pvc created
``` ```
**Note** deleting PV and PVC does not delete the backend CephFS subvolume, ### Verify CephFS static PVC
user needs to manually delete the CephFS subvolume if required.
We can verify the CephFS static PVC by creating a Pod that mounts the PVC.
```yaml
apiVersion: v1
kind: Pod
metadata:
name: cephfs-static-pvc-test
namespace: default
spec:
containers:
- name: busybox
image: busybox:latest
volumeMounts:
- name: static-pvc
mountPath: /data/pvc
command: ["sleep", "3600"]
volumes:
- name: static-pvc
persistentVolumeClaim:
claimName: cephfs-static-pvc
readOnly: false
```
```bash
$ kubectl create -f cephfs-static-pvc-test.yaml
pod/cephfs-static-pvc-test created
```
Verify that the PVC has been successfully mounted.
```bash
kubectl exec cephfs-static-pvc-test -- df -h /data/pvc
```
Once you have completed the verification step, you can delete the test Pod for CephFS.
```bash
$ kubectl delete pod cephfs-static-pvc-test
pod "cephfs-static-pvc-test" deleted
```
> [!note]
> deleting PV and PVC does not delete the backend CephFS subvolume or volume,
user needs to manually delete the CephFS subvolume or volume if required.