mirror of
https://github.com/ceph/ceph-csi.git
synced 2024-11-18 12:20:24 +00:00
144 lines
5.2 KiB
Markdown
144 lines
5.2 KiB
Markdown
|
# In-tree storage plugin to CSI Driver Migration
|
||
|
|
|
||
|
|
This document covers the example usage of in-tree RBD storage plugin to CSI
|
||
|
|
migration feature which can be enabled in a Kubernetes cluster. At present, this
|
||
|
|
feature is only supported for RBD in-tree plugin. Once this feature is enabled,
|
||
|
|
the in-tree volume requests (`kubernetes.io/rbd`) will be redirected to a
|
||
|
|
corresponding CSI (`rbd.csi.ceph.com`) driver.
|
||
|
|
|
||
|
|
## RBD
|
||
|
|
|
||
|
|
- [Prerequisite](#prerequisite)
|
||
|
|
- [Volume operations after enabling CSI migration](#volume-operations-after-enabling-csi-migration)
|
||
|
|
- [Create volume](#create-volume)
|
||
|
|
- [Mount volume to a POD](#mount-volume-to-a-pod)
|
||
|
|
- [Resize volume](#resize-volume)
|
||
|
|
- [Unmount volume](#unmount-volume)
|
||
|
|
- [Delete volume](#delete-volume)
|
||
|
|
- [References](#additional-references)
|
||
|
|
|
||
|
|
### Prerequisite
|
||
|
|
|
||
|
|
For in-tree RBD migration to CSI driver to be supported for your Kubernetes
|
||
|
|
cluster, the Kubernetes version running in your cluster should be >= v1.23. We
|
||
|
|
also need sidecar controllers (`csi-provisioner` and `csi-resizer`) which are
|
||
|
|
compatible with the Kubernetes version v1.23 to be available in your cluster.
|
||
|
|
You can enable the migration with a couple of feature gates in your Kubernetes
|
||
|
|
cluster. These feature gates are alpha in Kubernetes 1.23 release.
|
||
|
|
|
||
|
|
- `CSIMigrationRBD`: when enabled, it will redirect traffic from in-tree rbd
|
||
|
|
plugin (`kubernetes.io/rbd`) to CSI driver (`rbd.csi.ceph.com`), default
|
||
|
|
to `false` now.
|
||
|
|
|
||
|
|
- `IntreePluginRBDUnregister`: Disables the RBD in-tree driver
|
||
|
|
|
||
|
|
To enable feature gates, refer [feature gates](https://kubernetes.io/docs/reference/command-line-tools-reference/feature-gates/)
|
||
|
|
|
||
|
|
As a Kubernetes cluster operator that administers storage, here are the
|
||
|
|
prerequisites that you must complete before you attempt migration to the RBD CSI
|
||
|
|
driver:
|
||
|
|
|
||
|
|
- You must install the Ceph CSI driver (`rbd.csi.ceph.com`), v3.5.0 or above,
|
||
|
|
into your Kubernetes cluster.
|
||
|
|
- Configure `clusterID` field in the configmap as
|
||
|
|
discussed [here](https://github.com/ceph/ceph-csi/blob/devel/docs/design/proposals/intree-migrate.md#clusterid-field-in-the-migration-request)
|
||
|
|
- Configure migration secret as
|
||
|
|
discussed [here](https://github.com/ceph/ceph-csi/blob/devel/docs/design/proposals/intree-migrate.md#migration-secret-for-csi-operations)
|
||
|
|
|
||
|
|
In below examples, `fast-rbd` is in-tree storageclass with provisioner
|
||
|
|
referencing in-tree provisioner `Kubernetes.io/rbd`.
|
||
|
|
|
||
|
|
```console
|
||
|
|
$ kubectl describe sc fast-rbd |grep -i provisioner
|
||
|
|
Provisioner: Kubernetes.io/rbd
|
||
|
|
```
|
||
|
|
|
||
|
|
### Volume operations after enabling CSI migration
|
||
|
|
|
||
|
|
This section covers the operations on volumes provisioned after enabling CSI
|
||
|
|
migration in a cluster.
|
||
|
|
|
||
|
|
#### Create Volume
|
||
|
|
|
||
|
|
``` console
|
||
|
|
$ kubectl create -f pvc.yaml
|
||
|
|
persistentvolumeclaim/testpvc created
|
||
|
|
|
||
|
|
$ kubectl get pvc,pv
|
||
|
|
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
|
||
|
|
persistentvolumeclaim/testpvc Bound pvc-c4e7dca5-4be6-4168-8eb5-af6ade04261f 1Gi RWO fast-rbd 24s
|
||
|
|
|
||
|
|
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
|
||
|
|
persistentvolume/pvc-c4e7dca5-4be6-4168-8eb5-af6ade04261f 1Gi RWO Delete Bound default/testpvc fast-rbd 18s
|
||
|
|
```
|
||
|
|
|
||
|
|
#### Mount Volume to a POD
|
||
|
|
|
||
|
|
Create a pod with PVC and verify the mount inside the POD
|
||
|
|
|
||
|
|
```console
|
||
|
|
$ kubectl create -f pod.yaml
|
||
|
|
pod/task-pv-pod created
|
||
|
|
|
||
|
|
$ kubectl get pods
|
||
|
|
NAME READY STATUS RESTARTS AGE
|
||
|
|
task-pv-pod 1/1 Running 0 2m36s
|
||
|
|
|
||
|
|
$ kubectl get pvc
|
||
|
|
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
|
||
|
|
testpvc Bound pvc-c4e7dca5-4be6-4168-8eb5-af6ade04261f 1Gi RWO fast-rbd 4m40s
|
||
|
|
|
||
|
|
$ kubectl exec -ti task-pv-pod -- df -kh |grep nginx
|
||
|
|
/dev/rbd0 976M 2.6M 958M 1% /usr/share/nginx/html
|
||
|
|
```
|
||
|
|
|
||
|
|
#### Resize Volume
|
||
|
|
|
||
|
|
Resize PVC from 1Gi to 2Gi and verify the new size change in the POD
|
||
|
|
|
||
|
|
```console
|
||
|
|
$ kubectl patch pvc testpvc -p '{"spec":{"resources":{"requests":{"storage":"2Gi"}}}}}'
|
||
|
|
persistentvolumeclaim/testpvc patched
|
||
|
|
|
||
|
|
$ kubectl exec -ti task-pv-pod -- df -kh |grep nginx
|
||
|
|
/dev/rbd0 2.0G 3.0M 2.0G 1% /usr/share/nginx/html
|
||
|
|
```
|
||
|
|
|
||
|
|
#### Unmount Volume
|
||
|
|
|
||
|
|
Delete POD and verify pod deleted successfully
|
||
|
|
|
||
|
|
```console
|
||
|
|
$ kubectl get pods
|
||
|
|
NAME READY STATUS RESTARTS AGE
|
||
|
|
task-pv-pod 1/1 Running 0 5m31s
|
||
|
|
|
||
|
|
$ kubectl delete pod task-pv-pod
|
||
|
|
pod "task-pv-pod" deleted
|
||
|
|
|
||
|
|
$ kubectl get pods
|
||
|
|
```
|
||
|
|
|
||
|
|
#### Delete volume
|
||
|
|
|
||
|
|
Delete PVC and verify PVC and PV objects are deleted
|
||
|
|
|
||
|
|
```console
|
||
|
|
$ kubectl delete pvc testpvc
|
||
|
|
persistentvolumeclaim "testpvc" deleted
|
||
|
|
|
||
|
|
$ kubectl get pvc
|
||
|
|
No resources found in default namespace.
|
||
|
|
|
||
|
|
$ kubectl get pv
|
||
|
|
No resources found
|
||
|
|
```
|
||
|
|
|
||
|
|
### Additional References
|
||
|
|
|
||
|
|
To know more about in-tree to CSI migration:
|
||
|
|
|
||
|
|
- [design doc](./design/proposals/intree-migrate.md)
|
||
|
|
- [Kubernetes 1.17 Feature: Kubernetes In-Tree to CSI Volume Migration Moves to Beta](https://Kubernetes.io/blog/2019/12/09/Kubernetes-1-17-feature-csi-migration-beta/)
|
||
|
|
|