mirror of
https://github.com/ceph/ceph-csi.git
synced 2024-11-27 00:30:18 +00:00
388 lines
10 KiB
Markdown
388 lines
10 KiB
Markdown
# Dynamically Expand Volume
|
|
|
|
- [Dynamically Expand Volume](#dynamically-expand-volume)
|
|
- [Prerequisite](#prerequisite)
|
|
- [Expand RBD PVCs](#expand-rbd-pvcs)
|
|
- [Expand RBD Filesystem PVC](#expand-rbd-filesystem-pvc)
|
|
- [Expand RBD Block PVC](#expand-rbd-block-pvc)
|
|
- [Expand CephFS PVC](#expand-cephfs-pvc)
|
|
- [Expand CephFS Filesystem PVC](#expand-cephfs-filesystem-pvc)
|
|
|
|
## Prerequisite
|
|
|
|
- For filesystem expansion to be supported for your kubernetes cluster, the
|
|
kubernetes version running in your cluster should be >= v1.15 and for block
|
|
volume expand support the kubernetes version should be >=1.16. Also,
|
|
`ExpandCSIVolumes` feature gate has to be enabled for the volume expand
|
|
functionality to work.
|
|
|
|
- The controlling StorageClass must have `allowVolumeExpansion` set to `true`.
|
|
|
|
### Expand RBD PVCs
|
|
|
|
#### Expand RBD Filesystem PVC
|
|
|
|
pvc.yaml
|
|
|
|
```yaml
|
|
apiVersion: v1
|
|
items:
|
|
- apiVersion: v1
|
|
kind: PersistentVolumeClaim
|
|
metadata:
|
|
annotations:
|
|
pv.kubernetes.io/bind-completed: "yes"
|
|
pv.kubernetes.io/bound-by-controller: "yes"
|
|
volume.beta.kubernetes.io/storage-provisioner: rbd.csi.ceph.com
|
|
creationTimestamp: "2019-12-19T05:44:45Z"
|
|
finalizers:
|
|
- kubernetes.io/pvc-protection
|
|
name: rbd-pvc
|
|
namespace: default
|
|
resourceVersion: "3557"
|
|
selfLink: /api/v1/namespaces/default/persistentvolumeclaims/rbd-pvc
|
|
uid: efe688d6-a420-4041-900e-c5e19fd73ebf
|
|
spec:
|
|
accessModes:
|
|
- ReadWriteOnce
|
|
resources:
|
|
requests:
|
|
storage: 1Gi
|
|
storageClassName: csi-rbd-sc
|
|
volumeMode: Filesystem
|
|
volumeName: pvc-efe688d6-a420-4041-900e-c5e19fd73ebf
|
|
status:
|
|
accessModes:
|
|
- ReadWriteOnce
|
|
capacity:
|
|
storage: 1Gi
|
|
phase: Bound
|
|
kind: List
|
|
metadata:
|
|
resourceVersion: ""
|
|
selfLink: ""
|
|
```
|
|
|
|
- mounted Filesystem size in pod using this PVC
|
|
|
|
```bash
|
|
[$]kubectl exec -it csi-rbd-demo-pod sh
|
|
# df -h /var/lib/www/html
|
|
Filesystem Size Used Avail Use% Mounted on
|
|
/dev/rbd0 976M 2.6M 958M 1% /var/lib/www/html
|
|
```
|
|
|
|
- Now expand the PVC by editing the PVC (pvc.spec.resource.requests.storage)
|
|
|
|
```bash
|
|
[$]kubectl edit pvc rbd-pvc
|
|
```
|
|
|
|
Check PVC status after editing the pvc storage
|
|
|
|
```yaml
|
|
apiVersion: v1
|
|
items:
|
|
- apiVersion: v1
|
|
kind: PersistentVolumeClaim
|
|
metadata:
|
|
annotations:
|
|
pv.kubernetes.io/bind-completed: "yes"
|
|
pv.kubernetes.io/bound-by-controller: "yes"
|
|
volume.beta.kubernetes.io/storage-provisioner: rbd.csi.ceph.com
|
|
creationTimestamp: "2019-12-19T05:44:45Z"
|
|
finalizers:
|
|
- kubernetes.io/pvc-protection
|
|
name: rbd-pvc
|
|
namespace: default
|
|
resourceVersion: "4773"
|
|
selfLink: /api/v1/namespaces/default/persistentvolumeclaims/rbd-pvc
|
|
uid: efe688d6-a420-4041-900e-c5e19fd73ebf
|
|
spec:
|
|
accessModes:
|
|
- ReadWriteOnce
|
|
resources:
|
|
requests:
|
|
storage: 10Gi
|
|
storageClassName: csi-rbd-sc
|
|
volumeMode: Filesystem
|
|
volumeName: pvc-efe688d6-a420-4041-900e-c5e19fd73ebf
|
|
status:
|
|
accessModes:
|
|
- ReadWriteOnce
|
|
capacity:
|
|
storage: 1Gi
|
|
conditions:
|
|
- lastProbeTime: null
|
|
lastTransitionTime: "2019-12-19T05:49:39Z"
|
|
message: Waiting for user to (re-)start a pod to finish file system resize of
|
|
volume on node.
|
|
status: "True"
|
|
type: FileSystemResizePending
|
|
phase: Bound
|
|
kind: List
|
|
metadata:
|
|
resourceVersion: ""
|
|
selfLink: ""
|
|
```
|
|
|
|
Now you can see the PVC status as `FileSystemResizePending`, once the kubelet
|
|
calls the NodeExpandVolume to expand the PVC on node, the `status conditions`
|
|
and `status` will be updated
|
|
|
|
```bash
|
|
[$]kubectl get pvc
|
|
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
|
|
rbd-pvc Bound pvc-efe688d6-a420-4041-900e-c5e19fd73ebf 10Gi RWO csi-rbd-sc 7m6s
|
|
```
|
|
|
|
- Now let us check the directory size inside the pod where PVC is mounted
|
|
|
|
```bash
|
|
[$]kubectl exec -it csi-rbd-demo-pod sh
|
|
# df -h /var/lib/www/html
|
|
Filesystem Size Used Avail Use% Mounted on
|
|
/dev/rbd0 9.9G 4.5M 9.8G 1% /var/lib/www/html
|
|
```
|
|
|
|
now you can see the size of `/var/lib/www/html` is updated from 976M to 9.9G
|
|
|
|
#### Expand RBD Block PVC
|
|
|
|
```bash
|
|
[$]kubectl get pvc raw-block-pvc -o yaml
|
|
apiVersion: v1
|
|
kind: PersistentVolumeClaim
|
|
metadata:
|
|
annotations:
|
|
pv.kubernetes.io/bind-completed: "yes"
|
|
pv.kubernetes.io/bound-by-controller: "yes"
|
|
volume.beta.kubernetes.io/storage-provisioner: rbd.csi.ceph.com
|
|
creationTimestamp: "2019-12-19T05:56:02Z"
|
|
finalizers:
|
|
- kubernetes.io/pvc-protection
|
|
name: raw-block-pvc
|
|
namespace: default
|
|
resourceVersion: "6370"
|
|
selfLink: /api/v1/namespaces/default/persistentvolumeclaims/raw-block-pvc
|
|
uid: 54885275-7ca9-4b89-8e7e-c99f375d1174
|
|
spec:
|
|
accessModes:
|
|
- ReadWriteOnce
|
|
resources:
|
|
requests:
|
|
storage: 1Gi
|
|
storageClassName: csi-rbd-sc
|
|
volumeMode: Block
|
|
volumeName: pvc-54885275-7ca9-4b89-8e7e-c99f375d1174
|
|
status:
|
|
accessModes:
|
|
- ReadWriteOnce
|
|
capacity:
|
|
storage: 1Gi
|
|
phase: Bound
|
|
```
|
|
|
|
- Device size in pod using this PVC
|
|
|
|
```bash
|
|
[$]kubectl exec -it pod-with-raw-block-volume sh
|
|
sh-4.4# blockdev --getsize64 /dev/xvda
|
|
1073741824
|
|
```
|
|
|
|
rbd Block PVC is mounted at `/dev/xvda` of the pod, and the size is `1073741824`
|
|
bytes which is equal to `1Gib`
|
|
|
|
- Now expand the PVC
|
|
|
|
To expand PVC, change `(pvc.spec.resource.requests.storage)` to the new size
|
|
which should be greater than the current size.
|
|
|
|
```bash
|
|
[$]kubectl edit pvc raw-block-pvc
|
|
```
|
|
|
|
Check PVC status after editing the pvc storage
|
|
|
|
```yaml
|
|
apiVersion: v1
|
|
kind: PersistentVolumeClaim
|
|
metadata:
|
|
annotations:
|
|
pv.kubernetes.io/bind-completed: "yes"
|
|
pv.kubernetes.io/bound-by-controller: "yes"
|
|
volume.beta.kubernetes.io/storage-provisioner: rbd.csi.ceph.com
|
|
creationTimestamp: "2019-12-19T05:56:02Z"
|
|
finalizers:
|
|
- kubernetes.io/pvc-protection
|
|
name: raw-block-pvc
|
|
namespace: default
|
|
resourceVersion: "7923"
|
|
selfLink: /api/v1/namespaces/default/persistentvolumeclaims/raw-block-pvc
|
|
uid: 54885275-7ca9-4b89-8e7e-c99f375d1174
|
|
spec:
|
|
accessModes:
|
|
- ReadWriteOnce
|
|
resources:
|
|
requests:
|
|
storage: 10Gi
|
|
storageClassName: csi-rbd-sc
|
|
volumeMode: Block
|
|
volumeName: pvc-54885275-7ca9-4b89-8e7e-c99f375d1174
|
|
status:
|
|
accessModes:
|
|
- ReadWriteOnce
|
|
capacity:
|
|
storage: 1Gi
|
|
conditions:
|
|
- lastProbeTime: null
|
|
lastTransitionTime: "2019-12-19T06:02:15Z"
|
|
message: Waiting for user to (re-)start a pod to finish file system resize of
|
|
volume on node.
|
|
status: "True"
|
|
type: FileSystemResizePending
|
|
phase: Bound
|
|
```
|
|
|
|
Now you can see the PVC stats as `FileSystemResizePending`, once the kubelet calls
|
|
the NodeExpandVolume to expand the PVC on node, the status conditions will be updated
|
|
and `status.capacity.storage` will be updated.
|
|
|
|
```bash
|
|
[$]kubectl get pvc
|
|
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
|
|
raw-block-pvc Bound pvc-efe688d6-a420-4041-900e-c5e19fd73ebf 10Gi RWO csi-rbd-sc 7m6s
|
|
```
|
|
|
|
Device size in pod using this PVC
|
|
|
|
```bash
|
|
[$]kubectl exec -it pod-with-raw-block-volume sh
|
|
sh-4.4# blockdev --getsize64 /dev/xvda
|
|
10737418240
|
|
```
|
|
|
|
rbd Block PVC is mounted at `/dev/xvda` of the pod, and the size is
|
|
`10737418240` bytes which is equal to `10Gib`
|
|
|
|
### Expand CephFS PVC
|
|
|
|
#### Expand CephFS Filesystem PVC
|
|
|
|
pvc.yaml
|
|
|
|
```yaml
|
|
apiVersion: v1
|
|
items:
|
|
- apiVersion: v1
|
|
kind: PersistentVolumeClaim
|
|
metadata:
|
|
annotations:
|
|
pv.kubernetes.io/bind-completed: "yes"
|
|
pv.kubernetes.io/bound-by-controller: "yes"
|
|
volume.beta.kubernetes.io/storage-provisioner: cephfs.csi.ceph.com
|
|
creationTimestamp: "2020-01-17T07:55:11Z"
|
|
finalizers:
|
|
- kubernetes.io/pvc-protection
|
|
name: csi-cephfs-pvc
|
|
namespace: default
|
|
resourceVersion: "5955"
|
|
selfLink: /api/v1/namespaces/default/persistentvolumeclaims/csi-cephfs-pvc
|
|
uid: b84d07c9-ea67-40b4-96b9-4a79669b1ccc
|
|
spec:
|
|
accessModes:
|
|
- ReadWriteMany
|
|
resources:
|
|
requests:
|
|
storage: 5Gi
|
|
storageClassName: csi-cephfs-sc
|
|
volumeMode: Filesystem
|
|
volumeName: pvc-b84d07c9-ea67-40b4-96b9-4a79669b1ccc
|
|
status:
|
|
accessModes:
|
|
- ReadWriteMany
|
|
capacity:
|
|
storage: 5Gi
|
|
phase: Bound
|
|
kind: List
|
|
metadata:
|
|
resourceVersion: ""
|
|
selfLink: ""
|
|
```
|
|
|
|
- mounted Filesystem size in pod using this PVC
|
|
|
|
```bash
|
|
[$]kubectl exec -it csi-cephfs-demo-pod sh
|
|
# df -h /var/lib/www
|
|
Filesystem Size Used Avail Use% Mounted on
|
|
10.108.149.216:6789:/volumes/csi/csi-vol-b0a1bc79-38fe-11ea-adb6-1a2797ee96de 5.0G 0 5.0G 0% /var/lib/www
|
|
```
|
|
|
|
- Now expand the PVC by editing the PVC (pvc.spec.resource.requests.storage)
|
|
|
|
```bash
|
|
[$]kubectl edit pvc csi-cephfs-pvc
|
|
```
|
|
|
|
Check PVC status after editing the PVC storage
|
|
|
|
```yaml
|
|
apiVersion: v1
|
|
items:
|
|
- apiVersion: v1
|
|
kind: PersistentVolumeClaim
|
|
metadata:
|
|
annotations:
|
|
pv.kubernetes.io/bind-completed: "yes"
|
|
pv.kubernetes.io/bound-by-controller: "yes"
|
|
volume.beta.kubernetes.io/storage-provisioner: cephfs.csi.ceph.com
|
|
creationTimestamp: "2020-01-17T07:55:11Z"
|
|
finalizers:
|
|
- kubernetes.io/pvc-protection
|
|
name: csi-cephfs-pvc
|
|
namespace: default
|
|
resourceVersion: "6902"
|
|
selfLink: /api/v1/namespaces/default/persistentvolumeclaims/csi-cephfs-pvc
|
|
uid: b84d07c9-ea67-40b4-96b9-4a79669b1ccc
|
|
spec:
|
|
accessModes:
|
|
- ReadWriteMany
|
|
resources:
|
|
requests:
|
|
storage: 10Gi
|
|
storageClassName: csi-cephfs-sc
|
|
volumeMode: Filesystem
|
|
volumeName: pvc-b84d07c9-ea67-40b4-96b9-4a79669b1ccc
|
|
status:
|
|
accessModes:
|
|
- ReadWriteMany
|
|
capacity:
|
|
storage: 10Gi
|
|
phase: Bound
|
|
kind: List
|
|
metadata:
|
|
resourceVersion: ""
|
|
selfLink: ""
|
|
```
|
|
|
|
Now you can see the PVC status capacity storage is updated with request size
|
|
|
|
```bash
|
|
[$]kubectl get pvc
|
|
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
|
|
csi-cephfs-pvc Bound pvc-b84d07c9-ea67-40b4-96b9-4a79669b1ccc 10Gi RWX csi-cephfs-sc 6m26s
|
|
```
|
|
|
|
- Now let us check the directory size inside the pod where PVC is mounted
|
|
|
|
```bash
|
|
[$]kubectl exec -it csi-cephfs-demo-pod sh
|
|
# df -h /var/lib/www
|
|
Filesystem Size Used Avail Use% Mounted on
|
|
10.108.149.216:6789:/volumes/csi/csi-vol-b0a1bc79-38fe-11ea-adb6-1a2797ee96de 10G 0 10G 0% /var/lib/www
|
|
```
|
|
|
|
now you can see the size of `/var/lib/www` is updated from 5G to 10G
|