From 00c12b396f37de8824978c99fd03cfd8afd67c0f Mon Sep 17 00:00:00 2001 From: Praveen M Date: Fri, 17 Nov 2023 12:00:19 +0530 Subject: [PATCH] doc: add documentation for read affinity This commit adds documentation about read affinity supported for CephFS subvolumes. Signed-off-by: Praveen M --- charts/ceph-csi-cephfs/README.md | 2 +- docs/deploy-cephfs.md | 23 +++++++++++++++++++++++ 2 files changed, 24 insertions(+), 1 deletion(-) diff --git a/charts/ceph-csi-cephfs/README.md b/charts/ceph-csi-cephfs/README.md index 31678e9ea..2bfe4df53 100644 --- a/charts/ceph-csi-cephfs/README.md +++ b/charts/ceph-csi-cephfs/README.md @@ -165,7 +165,7 @@ charts and their default values. | `provisionerSocketFile` | The filename of the provisioner socket | `csi-provisioner.sock` | | `pluginSocketFile` | The filename of the plugin socket | `csi.sock` | | `readAffinity.enabled` | Enable read affinity for CephFS subvolumes. Recommended to set to true if running kernel 5.8 or newer. | `false` | -| `readAffinity.crushLocationLabels` | Define which node labels to use as CRUSH location. This should correspond to the values set in the CRUSH map. For more information, click [here](https://github.com/ceph/ceph-csi/blob/v3.9.0/docs/deploy-rbd.md#read-affinity-using-crush-locations-for-rbd-volumes)| `[]` | +| `readAffinity.crushLocationLabels` | Define which node labels to use as CRUSH location. This should correspond to the values set in the CRUSH map. For more information, click [here](https://github.com/ceph/ceph-csi/blob/v3.9.0/docs/deploy-cephfs.md#read-affinity-using-crush-locations-for-cephfs-subvolumes)| `[]` | | `kubeletDir` | Kubelet working directory | `/var/lib/kubelet` | | `driverName` | Name of the csi-driver | `cephfs.csi.ceph.com` | | `configMapName` | Name of the configmap which contains cluster configuration | `ceph-csi-config` | diff --git a/docs/deploy-cephfs.md b/docs/deploy-cephfs.md index 48043877a..ddb3f40f3 100644 --- a/docs/deploy-cephfs.md +++ b/docs/deploy-cephfs.md @@ -47,6 +47,8 @@ make image-cephcsi | `--kernelmountoptions` | _empty_ | Comma separated string of mount options accepted by cephfs kernel mounter.
`Note: These options will be replaced if kernelMountOptions are defined in the ceph-csi-config ConfigMap for the specific cluster.` | | `--fusemountoptions` | _empty_ | Comma separated string of mount options accepted by ceph-fuse mounter.
`Note: These options will be replaced if fuseMountOptions are defined in the ceph-csi-config ConfigMap for the specific cluster.` | | `--domainlabels` | _empty_ | Kubernetes node labels to use as CSI domain labels for topology aware provisioning, should be a comma separated value (ex:= "failure-domain/region,failure-domain/zone") | +| `--enable-read-affinity` | `false` | enable read affinity | +| `--crush-location-labels`| _empty_ | Kubernetes node labels that determine the CRUSH location the node belongs to, separated by ','.
`Note: These labels will be replaced if crush location labels are defined in the ceph-csi-config ConfigMap for the specific cluster.` | **NOTE:** The parameter `-forcecephkernelclient` enables the Kernel CephFS mounter on kernels < 4.17. @@ -223,6 +225,27 @@ The Helm chart is located in `charts/ceph-csi-cephfs`. [See the Helm chart readme for installation instructions.](../charts/ceph-csi-cephfs/README.md) +## Read Affinity using crush locations for CephFS subvolumes + +Ceph CSI supports mounting CephFS subvolumes with kernel mount options +`"read_from_replica=localize,crush_location=type1:value1|type2:value2"` to +allow serving reads from the most local OSD (according to OSD locations as +defined in the CRUSH map). + +This can be enabled by adding labels to Kubernetes nodes like +`"topology.io/region=east"` and `"topology.io/zone=east-zone1"` and +passing command line arguments `"--enable-read-affinity=true"` and +`"--crush-location-labels=topology.io/zone,topology.io/region"` to Ceph CSI +CephFS daemonset pod "csi-cephfsplugin" container, resulting in Ceph CSI adding +`"--options read_from_replica=localize,crush_location=zone:east-zone1|region:east"` +kernel mount options during cephfs mount operation. +If enabled, this option will be added to all CephFS subvolumes mapped by Ceph CSI. +Well known labels can be found +[here](https://kubernetes.io/docs/reference/labels-annotations-taints/). + +>Note: Label values will have all its dots `"."` normalized with dashes `"-"` +in order for it to work with ceph CRUSH map. + ## CephFS Volume Encryption Requires fscrypt support in the Linux kernel and Ceph.