updated README

2024-12-18 02:50:30 +00:00 · 2018-03-26 15:02:34 +02:00 · 2018-03-26 15:02:34 +02:00 · 9cc4862856
commit 9cc4862856
parent 48b4177949
1 changed files with 154 additions and 15 deletions
--- a/README.md
+++ b/README.md
@ -2,21 +2,22 @@

 ## Overview

-RBD CSI plugin implements an interface between  CSI enabled Container
+Ceph CSI plugins implement an interface between CSI enabled Container
 Orchestrator and CEPH cluster. It allows dynamically provision CEPH
 volumes and attach it to workloads.
-Current implementation of CSI RBD plugin was tested in Kubernetes environment,
-but its code does not rely on any Kubernetes specific calls (WIP to make it k8s agnostic)
+Current implementation of Ceph CSI plugins was tested in Kubernetes environment (requires Kubernetes 1.10+),
+but the code does not rely on any Kubernetes specific calls (WIP to make it k8s agnostic)
 and should be able to run with any CSI enabled CO (Containers Orchestration).

+[Container Storage Interface (CSI)](https://github.com/container-storage-interface/) driver, provisioner, and attacher for Ceph RBD and CephFS
+
+## RBD Plugin
+
 An RBD CSI plugin is available to help simplify storage management.
 Once user creates PVC with the reference to a RBD storage class, rbd image and 
 corresponding PV object gets dynamically created and becomes ready to be used by
 workloads.

-[Container Storage Interface (CSI)](https://github.com/container-storage-interface/) driver, provisioner, and attacher for Ceph RBD and CephFS
-
-## RBD Plugin
 ### Configuration Requirements

 * Secret object with the authentication key for ceph cluster
@ -44,7 +45,7 @@ $ make rbdplugin

 To build a container:
 ```
-$ make container
+$ make container-rbdplugin
 ```
 By running:
 ```
@ -73,29 +74,29 @@ Enable features `MountPropagation=true,CSIPersistentVolume=true` and runtime con

 #### Step 1: Create Secret 
 ```
-$ kubectl create -f ./deploy/kubernetes/rbd-secrets.yaml 
+$ kubectl create -f ./deploy/rbd/kubernetes/rbd-secrets.yaml 
 ```
 **Important:** rbd-secrets.yaml, must be customized to match your ceph environment.

 #### Step 2: Create StorageClass
 ```
-$ kubectl create -f ./deploy/kubernetes/rbd-storage-class.yaml
+$ kubectl create -f ./deploy/rbd/kubernetes/rbd-storage-class.yaml
 ```
 **Important:** rbd-storage-class.yaml, must be customized to match your ceph environment.

 #### Step 3: Start CSI CEPH RBD plugin
 ```
-$ kubectl create -f ./deploy/kubernetes/rbdplugin.yaml
+$ kubectl create -f ./deploy/rbd/kubernetes/rbdplugin.yaml
 ```

 #### Step 4: Start CSI External Attacher
 ```
-$ kubectl create -f ./deploy/kubernetes/csi-attacher.yaml
+$ kubectl create -f ./deploy/rbd/kubernetes/csi-attacher.yaml
 ```

 #### Step 5: Start CSI External Provisioner  
 ```
-$ kubectl create -f ./deploy/kubernetes/csi-provisioner.yaml
+$ kubectl create -f ./deploy/rbd/kubernetes/csi-provisioner.yaml
 ```
 **Important:** Deployment yaml files includes required Service Account definitions and
 required RBAC rules.
@ -116,7 +117,7 @@ default       csi-provisioner-0                                   1/1       Runn

 #### Step 7: Create PVC 
 ```
-$ kubectl create -f ./deploy/kubernetes/pvc.yaml
+$ kubectl create -f ./deploy/rbd/kubernetes/pvc.yaml
 ```

 #### Step 8: Check status of provisioner PV  
@ -152,12 +153,150 @@ Source:
 #### Step 9: Create a test pod

 ```bash
-# kubectl create -f ./deploy/pod.yaml
+# kubectl create -f ./deploy/rbd/pod.yaml
 ```

 ## CephFS plugin

-TODO 
+A CephFS CSI plugin is available to help simplify storage management.
+Once user creates PVC with the reference to a CephFS CSI storage class, corresponding
+PV object gets dynamically created and becomes ready to be used by workloads.
+
+### Configuration Requirements
+
+* Secret object with the authentication key for ceph cluster
+* StorageClass with csi-cephfsplugin (default CSI CephFS plugin name) as a provisioner name
+  and information about ceph cluster (mounter, monitors, user name, secret)
+* CephFS root path, which is then mounted by either `ceph-fuse` or `mount.ceph` - configurable
+  through the StorageClass
+* Service Accounts with required RBAC permissions   
+
+### Feature Status
+
+### 1.10: Alpha
+
+**Important:** `CSIPersistentVolume` and `MountPropagation`
+[feature gates must be enabled starting in 1.9](#enabling-the-alpha-feature-gates).
+Also API server must run with running config set to: `storage.k8s.io/v1alpha1`
+
+* `kube-apiserver` must be launched with `--feature-gate=CSIPersistentVolume=true,MountPropagation=true`
+  and `--runtime-config=storage.k8s.io/v1alpha1=true`
+* `kube-controller-manager` must be launched with `--feature-gate=CSIPersistentVolume=true`
+* `kubelet` must be launched with `--feature-gate=CSIPersistentVolume=true,MountPropagation=true`
+
+### Compiling
+CSI CephFS plugin can be compiled in a form of a binary file or in a form of a container. When compiled
+as a binary file, it gets stored in \_output folder with the name cephfsplugin. When compiled as a container,
+the resulting image is stored in a local docker's image store. 
+
+To compile just a binary file:
+```
+$ make cephfsplugin
+```
+
+To build a container:
+```
+$ make cephfsplugin-container
+```
+By running:
+```
+$ docker images | grep cephfsplugin
+```
+You should see the following line in the output:
+```
+quay.io/cephcsi/cephfsplugin              v0.2.0                            79482e644593        4 minutes ago       305MB
+```
+
+### Testing
+
+#### Prerequisite
+
+##### Enable Mount Propagation in Docker 
+
+Comment out `MountFlags=slave` in docker systemd service then restart docker service.
+```
+# systemctl daemon-reload
+# systemctl restart docker
+```
+
+##### Enable Kubernetes Feature Gates
+
+Enable features `MountPropagation=true,CSIPersistentVolume=true` and runtime config `storage.k8s.io/v1alpha1=true`
+
+#### Step 1: Create Secret 
+```
+$ kubectl create -f ./deploy/cephfs/kubernetes/secret.yaml
+```
+**Important:** secret.yaml, must be customized to match your ceph environment.
+
+#### Step 2: Create StorageClass
+```
+$ kubectl create -f ./deploy/cephfs/kubernetes/cephfs-storage-class.yaml
+```
+**Important:** cephfs-storage-class.yaml, must be customized to match your ceph environment.
+
+#### Step 3: Start CSI CEPH CephFS plugin
+```
+$ kubectl create -f ./deploy/cephfs/kubernetes/cephfsplugin.yaml
+```
+
+#### Step 4: Start CSI External Attacher
+```
+$ kubectl create -f ./deploy/cephfs/kubernetes/csi-attacher.yaml
+```
+
+#### Step 5: Start CSI External Provisioner  
+```
+$ kubectl create -f ./deploy/cephfs/kubernetes/csi-provisioner.yaml
+```
+**Important:** Deployment yaml files includes required Service Account definitions and
+required RBAC rules.
+
+#### Step 6: Check status of CSI CephFS plugin  
+```
+$ kubectl get pods | grep csi 
+csi-attacher-0           1/1       Running   0          6m
+csi-cephfsplugin-hmqpk   2/2       Running   0          6m
+csi-provisioner-0        1/1       Running   0          6m
+```
+
+#### Step 7: Create PVC 
+```
+$ kubectl create -f ./deploy/cephfs/kubernetes/pvc.yaml
+```
+
+#### Step 8: Check status of provisioner PV  
+```
+$ kubectl get pv
+NAME                                     CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS    CLAIM                    STORAGECLASS   REASON    AGE
+kubernetes-dynamic-pv-715cef0b30d811e8   5Gi        RWX            Delete           Bound     default/csi-cephfs-pvc   csi-cephfs               5s
+```
+
+```
+$ kubectl describe pv kubernetes-dynamic-pv-715cef0b30d811e8
+Name:            kubernetes-dynamic-pv-715cef0b30d811e8
+Labels:          <none>
+Annotations:     pv.kubernetes.io/provisioned-by=csi-cephfsplugin
+StorageClass:    csi-cephfs
+Status:          Bound
+Claim:           default/csi-cephfs-pvc
+Reclaim Policy:  Delete
+Access Modes:    RWX
+Capacity:        5Gi
+Message:         
+Source:
+    Type:    CSI (a Container Storage Interface (CSI) volume source)
+    Driver:      ReadOnly:  %v
+
+    VolumeHandle:                                                                    csi-cephfsplugin
+%!(EXTRA string=csi-cephfs-7182b779-30d8-11e8-bf01-5254007d7491, bool=false)Events:  <none>
+```
+
+#### Step 9: Create a test pod
+
+```
+$ kubectl create -f ./deploy/cephfs/kubernetes/pod.yaml
+```

 ## Troubleshooting