ceph-csi/docs/intree-migrate.md

In-tree storage plugin to CSI Driver Migration

This document covers the example usage of in-tree RBD storage plugin to CSI migration feature which can be enabled in a Kubernetes cluster. At present, this feature is only supported for RBD in-tree plugin. Once this feature is enabled, the in-tree volume requests (kubernetes.io/rbd) will be redirected to a corresponding CSI (rbd.csi.ceph.com) driver.

RBD

• Prerequisite
• Volume operations after enabling CSI migration
  • Create volume
  • Mount volume to a POD
  • Resize volume
  • Unmount volume
  • Delete volume
• References

Prerequisite

For in-tree RBD migration to CSI driver to be supported for your Kubernetes cluster, the Kubernetes version running in your cluster should be >= v1.23. We also need sidecar controllers (csi-provisioner and csi-resizer) which are compatible with the Kubernetes version v1.23 to be available in your cluster. You can enable the migration with a couple of feature gates in your Kubernetes cluster. These feature gates are alpha in Kubernetes 1.23 release.

• CSIMigrationRBD: when enabled, it will redirect traffic from in-tree rbd plugin (kubernetes.io/rbd) to CSI driver (rbd.csi.ceph.com), default to false now.

• IntreePluginRBDUnregister: Disables the RBD in-tree driver

To enable feature gates, refer feature gates

As a Kubernetes cluster operator that administers storage, here are the prerequisites that you must complete before you attempt migration to the RBD CSI driver:

• You must install the Ceph CSI driver (rbd.csi.ceph.com), v3.5.0 or above, into your Kubernetes cluster.
• Configure clusterID field in the configmap as discussed here
• Configure migration secret as discussed here

In below examples, fast-rbd is in-tree storageclass with provisioner referencing in-tree provisioner Kubernetes.io/rbd.

$ kubectl describe sc fast-rbd |grep -i provisioner
Provisioner:           Kubernetes.io/rbd

Volume operations after enabling CSI migration

This section covers the operations on volumes provisioned after enabling CSI migration in a cluster.

Create Volume

$ kubectl create -f pvc.yaml
persistentvolumeclaim/testpvc created

$ kubectl get pvc,pv
NAME                               STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
persistentvolumeclaim/testpvc      Bound    pvc-c4e7dca5-4be6-4168-8eb5-af6ade04261f   1Gi        RWO            fast-rbd       24s

NAME                                                        CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM             STORAGECLASS   REASON   AGE
persistentvolume/pvc-c4e7dca5-4be6-4168-8eb5-af6ade04261f   1Gi        RWO            Delete           Bound    default/testpvc   fast-rbd                18s

Mount Volume to a POD

Create a pod with PVC and verify the mount inside the POD

$ kubectl create -f pod.yaml
pod/task-pv-pod created

$ kubectl get pods
NAME                                         READY   STATUS    RESTARTS   AGE
task-pv-pod                                  1/1     Running   0          2m36s

$ kubectl get pvc
NAME         STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
testpvc      Bound    pvc-c4e7dca5-4be6-4168-8eb5-af6ade04261f   1Gi        RWO            fast-rbd       4m40s

$ kubectl exec -ti task-pv-pod -- df -kh |grep nginx
/dev/rbd0                                976M  2.6M  958M   1% /usr/share/nginx/html

Resize Volume

Resize PVC from 1Gi to 2Gi and verify the new size change in the POD

$ kubectl patch pvc testpvc -p '{"spec":{"resources":{"requests":{"storage":"2Gi"}}}}}'
persistentvolumeclaim/testpvc patched

$ kubectl exec -ti task-pv-pod -- df -kh |grep nginx
/dev/rbd0                                2.0G  3.0M  2.0G   1% /usr/share/nginx/html

Unmount Volume

Delete POD and verify pod deleted successfully

$ kubectl get pods
NAME                                         READY   STATUS    RESTARTS   AGE
task-pv-pod                                  1/1     Running   0          5m31s

$ kubectl delete pod task-pv-pod
pod "task-pv-pod" deleted

$ kubectl get pods

Delete volume

Delete PVC and verify PVC and PV objects are deleted

$ kubectl delete pvc testpvc
persistentvolumeclaim "testpvc" deleted

$ kubectl get pvc
No resources found in default namespace.

$ kubectl get pv
No resources found

Additional References

To know more about in-tree to CSI migration:

• design doc
• Kubernetes 1.17 Feature: Kubernetes In-Tree to CSI Volume Migration Moves to Beta