02bcb5f16a
Use Deployment with leader election instead of StatefulSet Deployment behaves better when a node gets disconnected from the rest of the cluster - new provisioner leader is elected in ~15 seconds, while it may take up to 5 minutes for StatefulSet to start a new replica. Refer: kubernetes-csi/external-provisioner@52d1fbc Signed-off-by: Madhu Rajanna <madhupr007@gmail.com> |
||
|---|---|---|
| .. | ||
| cephfs.go | ||
| deploy-rook.go | ||
| e2e_test.go | ||
| rbd.go | ||
| README.md | ||
| utils.go | ||
End-to-End Testing
Introduction
End-to-end (e2e) in cephcsi provides a mechanism to test the end-to-end behavior of the system, These tests will interact with live instances of ceph cluster just like how a user would.
The primary objectives of the e2e tests are to ensure a consistent and reliable behavior of the cephcsi code base and to catch hard-to-test bugs before users do when unit and integration tests are insufficient.
The Test framework is designed to install Rook, run cephcsi tests, and uninstall Rook.
The e2e test are built on top of Ginkgo and Gomega
Install Kubernetes
The cephcsi also provides a script for starting Kubernetes using minikube so users can quickly spin up a Kubernetes cluster.
the following parameters are available to configure kubernetes cluster
| flag | description |
|---|---|
| up | Starts a local kubernetes cluster and prepare a disk for rook |
| down | Stops a running local kubernetes cluster |
| clean | Deletes a local kubernetes cluster |
| ssh | Log into or run a command on a minikube machine with SSH |
| cephcsi | Copy built docker images to kubernetes cluster |
| k8s-sidecar | Copy kubernetes sidecar docker images to kubernetes cluster |
following environment variables can be exported to customize kubernetes deployment
| ENV | Description | Default |
|---|---|---|
| MINIKUBE_VERSION | minikube version to install | latest |
| KUBE_VERSION | kubernetes version to install | v1.13.0 |
| MEMORY | Amount of RAM allocated to the minikube VM in MB | 3000 |
| VM_DRIVER | VM driver to create virtual machine | virtualbox |
| CEPHCSI_IMAGE_REPO | Repo URL to pull cephcsi images | quay.io/cephcsi |
| K8S_IMAGE_REPO | Repo URL to pull kubernetes sidecar images | quay.io/k8scsi |
| K8S_FEATURE_GATES | Feature gates to enable on kubernetes cluster | BlockVolume=true,CSIBlockVolume=true,VolumeSnapshotDataSource=true |
- creating kubernetes cluster
$./minikube.sh up
- Teardown kubernetes cluster
$./minikube.sh clean
Test parameters
In addition to standard go tests parameters, the following custom parameters are available while running tests:
| flag | description |
|---|---|
| rook-version | Rook version to pull yaml files to deploy rook operator (default: master) |
| deploy-rook | Deploy rook operator to create ceph cluster (default: true) |
| deploy-timeout | Timeout to wait for created kubernetes resources (default: 10) |
| kubeconfig | Path to kubeconfig containing embedded authinfo (default: $HOME/.kube/config) |
| timeout | Panic test binary after duration d (default 0, timeout disabled) |
| v | Verbose: print additional output |
Running E2E
Note:- Prior to running the tests, you may need to copy the kubernetes configuration file to$HOME/.kube/configwhich is required to communicate with kubernetes cluster or you can passkubeconfigflag while running tests.
Functional tests are run by the go test command.
$go test ./e2e/ --rook-version="v1.0.1" --deploy-rook=true -timeout=20m -v
Functional tests can be invoked by make command
$make func-test TESTOPTIONS="--rook-version=v1.0.1 --deploy-rook=true --deploy-timeout=10 -timeout=30m -v"