2019-04-25 19:12:36 +00:00
|
|
|
# Ceph CSI
|
2018-01-20 20:04:38 +00:00
|
|
|
|
2021-08-19 10:32:48 +00:00
|
|
|
[](https://github.com/ceph/ceph-csi/releases)
|
2022-05-02 12:32:04 +00:00
|
|
|
[](https://mergify.com)
|
2019-04-04 06:49:06 +00:00
|
|
|
[](https://goreportcard.com/report/github.com/ceph/ceph-csi)
|
2021-08-19 08:44:34 +00:00
|
|
|
[](https://www.tickgit.com/browse?repo=github.com/ceph/ceph-csi&branch=devel)
|
2022-04-12 14:17:59 +00:00
|
|
|
[](https://bestpractices.coreinfrastructure.org/projects/5940)
|
2019-04-04 06:49:06 +00:00
|
|
|
|
2019-07-30 14:20:22 +00:00
|
|
|
- [Ceph CSI](#ceph-csi)
|
2022-11-09 13:37:26 +00:00
|
|
|
- [Overview](#overview)
|
|
|
|
|
- [Project status](#project-status)
|
|
|
|
|
- [Known to work CO platforms](#known-to-work-co-platforms)
|
|
|
|
|
- [Support Matrix](#support-matrix)
|
|
|
|
|
- [Ceph-CSI features and available versions](#ceph-csi-features-and-available-versions)
|
|
|
|
|
- [CSI spec and Kubernetes version compatibility](#csi-spec-and-kubernetes-version-compatibility)
|
|
|
|
|
- [Ceph CSI Container images and release compatibility](#ceph-csi-container-images-and-release-compatibility)
|
|
|
|
|
- [Contributing to this repo](#contributing-to-this-repo)
|
|
|
|
|
- [Troubleshooting](#troubleshooting)
|
|
|
|
|
- [Weekly Bug Triage call](#weekly-bug-triage-call)
|
|
|
|
|
- [Dev standup](#dev-standup)
|
|
|
|
|
- [Contact](#contact)
|
2019-07-12 07:32:44 +00:00
|
|
|
|
2022-07-31 08:24:26 +00:00
|
|
|
This repo contains the Ceph
|
2019-07-22 21:50:36 +00:00
|
|
|
[Container Storage Interface (CSI)](https://github.com/container-storage-interface/)
|
2022-07-31 08:24:26 +00:00
|
|
|
driver for RBD, CephFS and Kubernetes sidecar deployment YAMLs to support CSI
|
|
|
|
|
functionality: provisioner, attacher, resizer, driver-registrar and snapshotter.
|
2018-02-06 18:30:05 +00:00
|
|
|
|
2018-07-31 09:31:12 +00:00
|
|
|
## Overview
|
2018-03-26 13:02:34 +00:00
|
|
|
|
2022-07-31 08:24:26 +00:00
|
|
|
Ceph CSI plugins implement an interface between a CSI-enabled Container Orchestrator
|
|
|
|
|
(CO) and Ceph clusters. They enable dynamically provisioning Ceph volumes and
|
2019-04-25 19:12:36 +00:00
|
|
|
attaching them to workloads.
|
2018-03-26 13:02:34 +00:00
|
|
|
|
2019-04-25 19:12:36 +00:00
|
|
|
Independent CSI plugins are provided to support RBD and CephFS backed volumes,
|
2018-03-26 13:02:34 +00:00
|
|
|
|
2019-06-10 08:37:24 +00:00
|
|
|
- For details about configuration and deployment of RBD plugin, please refer
|
2024-11-12 20:42:00 +00:00
|
|
|
[rbd doc](https://github.com/ceph/ceph-csi/blob/devel/docs/rbd/deploy.md) and
|
2019-06-10 08:37:24 +00:00
|
|
|
for CephFS plugin configuration and deployment please
|
2024-11-12 20:42:00 +00:00
|
|
|
refer [cephFS doc](https://github.com/ceph/ceph-csi/blob/devel/docs/cephfs/deploy.md).
|
2022-07-31 08:24:26 +00:00
|
|
|
- For example usage of the RBD and CephFS CSI plugins, see examples in `examples/`.
|
2020-04-03 07:10:29 +00:00
|
|
|
- Stale resource cleanup, please refer [cleanup doc](docs/resource-cleanup.md).
|
2019-04-25 19:12:36 +00:00
|
|
|
|
2019-11-07 09:55:12 +00:00
|
|
|
NOTE:
|
|
|
|
|
|
|
|
|
|
- Ceph CSI **`Arm64`** support is experimental.
|
|
|
|
|
|
2019-04-25 19:12:36 +00:00
|
|
|
## Project status
|
|
|
|
|
|
2019-07-12 07:32:44 +00:00
|
|
|
Status: **GA**
|
2019-04-25 19:12:36 +00:00
|
|
|
|
2021-09-02 06:02:51 +00:00
|
|
|
## Known to work CO platforms
|
2019-04-25 19:12:36 +00:00
|
|
|
|
2022-07-31 08:24:26 +00:00
|
|
|
Ceph CSI drivers are currently developed and tested **exclusively** in Kubernetes
|
2021-09-02 06:02:51 +00:00
|
|
|
environments.
|
|
|
|
|
|
2023-11-27 09:44:19 +00:00
|
|
|
| Ceph CSI Version | Container Orchestrator Name | Version Tested |
|
|
|
|
|
| -----------------| --------------------------- | -------------------|
|
2024-12-03 07:09:26 +00:00
|
|
|
| v3.13.0 | Kubernetes | v1.29, v1.30, v1.31|
|
|
|
|
|
| v3.12.3 | Kubernetes | v1.29, v1.30, v1.31|
|
2021-09-02 06:02:51 +00:00
|
|
|
|
2022-07-31 08:24:26 +00:00
|
|
|
There is work in progress to make this CO-independent and thus
|
|
|
|
|
support other orchestration environments (Nomad, Mesos..etc).
|
2019-04-25 19:12:36 +00:00
|
|
|
|
2019-07-12 07:32:44 +00:00
|
|
|
NOTE:
|
2019-04-25 19:12:36 +00:00
|
|
|
|
2022-07-31 08:24:26 +00:00
|
|
|
The supported window of Ceph CSI versions is "N.(x-1)":
|
2021-08-25 09:37:49 +00:00
|
|
|
(N (Latest major release) . (x (Latest minor release) - 1)).
|
|
|
|
|
|
2023-06-26 05:13:15 +00:00
|
|
|
For example, if the Ceph CSI latest major version is `3.9.0` today, support is
|
|
|
|
|
provided for the versions above `3.8.0`. If users are running an unsupported
|
2022-07-31 08:24:26 +00:00
|
|
|
Ceph CSI version, they will be asked to upgrade when requesting support.
|
2018-01-20 20:04:38 +00:00
|
|
|
|
2019-05-08 08:18:53 +00:00
|
|
|
## Support Matrix
|
|
|
|
|
|
|
|
|
|
### Ceph-CSI features and available versions
|
|
|
|
|
|
2024-11-12 20:42:00 +00:00
|
|
|
Please refer [rbd nbd mounter](./docs/design/proposals/rbd-nbd.md#support-matrix)
|
2021-07-12 07:42:42 +00:00
|
|
|
for its support details.
|
|
|
|
|
|
2020-07-23 06:34:20 +00:00
|
|
|
| Plugin | Features | Feature Status | CSI Driver Version | CSI Spec Version | Ceph Cluster Version | Kubernetes Version |
|
|
|
|
|
| ------ | --------------------------------------------------------- | -------------- | ------------------ | ---------------- | -------------------- | ------------------ |
|
2023-11-17 09:11:58 +00:00
|
|
|
| RBD | Dynamically provision, de-provision Block mode RWO volume | GA | >= v1.0.0 | >= v1.0.0 | Pacific (>=v16.2.0) | >= v1.14.0 |
|
|
|
|
|
| | Dynamically provision, de-provision Block mode RWX volume | GA | >= v1.0.0 | >= v1.0.0 | Pacific (>=v16.2.0) | >= v1.14.0 |
|
|
|
|
|
| | Dynamically provision, de-provision Block mode RWOP volume| Alpha | >= v3.5.0 | >= v1.5.0 | Pacific (>=v16.2.0) | >= v1.22.0 |
|
|
|
|
|
| | Dynamically provision, de-provision File mode RWO volume | GA | >= v1.0.0 | >= v1.0.0 | Pacific (>=v16.2.0) | >= v1.14.0 |
|
|
|
|
|
| | Dynamically provision, de-provision File mode RWOP volume | Alpha | >= v3.5.0 | >= v1.5.0 | Pacific (>=v16.2.0) | >= v1.22.0 |
|
|
|
|
|
| | Provision File Mode ROX volume from snapshot | Alpha | >= v3.0.0 | >= v1.0.0 | Pacific (>=v16.2.0) | >= v1.17.0 |
|
|
|
|
|
| | Provision File Mode ROX volume from another volume | Alpha | >= v3.0.0 | >= v1.0.0 | Pacific (>=v16.2.0) | >= v1.16.0 |
|
|
|
|
|
| | Provision Block Mode ROX volume from snapshot | Alpha | >= v3.0.0 | >= v1.0.0 | Pacific (>=v16.2.0) | >= v1.17.0 |
|
|
|
|
|
| | Provision Block Mode ROX volume from another volume | Alpha | >= v3.0.0 | >= v1.0.0 | Pacific (>=v16.2.0) | >= v1.16.0 |
|
|
|
|
|
| | Creating and deleting snapshot | GA | >= v1.0.0 | >= v1.0.0 | Pacific (>=v16.2.0) | >= v1.17.0 |
|
|
|
|
|
| | Provision volume from snapshot | GA | >= v1.0.0 | >= v1.0.0 | Pacific (>=v16.2.0) | >= v1.17.0 |
|
|
|
|
|
| | Provision volume from another volume | GA | >= v1.0.0 | >= v1.0.0 | Pacific (>=v16.2.0) | >= v1.16.0 |
|
|
|
|
|
| | Expand volume | Beta | >= v2.0.0 | >= v1.1.0 | Pacific (>=v16.2.0) | >= v1.15.0 |
|
|
|
|
|
| | Volume/PV Metrics of File Mode Volume | GA | >= v1.2.0 | >= v1.1.0 | Pacific (>=v16.2.0) | >= v1.15.0 |
|
|
|
|
|
| | Volume/PV Metrics of Block Mode Volume | GA | >= v1.2.0 | >= v1.1.0 | Pacific (>=v16.2.0) | >= v1.21.0 |
|
|
|
|
|
| | Topology Aware Provisioning Support | Alpha | >= v2.1.0 | >= v1.1.0 | Pacific (>=v16.2.0) | >= v1.14.0 |
|
|
|
|
|
| CephFS | Dynamically provision, de-provision File mode RWO volume | GA | >= v1.1.0 | >= v1.0.0 | Pacific (>=v16.2.0) | >= v1.14.0 |
|
|
|
|
|
| | Dynamically provision, de-provision File mode RWX volume | GA | >= v1.1.0 | >= v1.0.0 | Pacific (>=v16.2.0) | >= v1.14.0 |
|
|
|
|
|
| | Dynamically provision, de-provision File mode ROX volume | Alpha | >= v3.0.0 | >= v1.0.0 | Pacific (>=v16.2.0) | >= v1.14.0 |
|
|
|
|
|
| | Dynamically provision, de-provision File mode RWOP volume | Alpha | >= v3.5.0 | >= v1.5.0 | Pacific (>=v16.2.0) | >= v1.22.0 |
|
|
|
|
|
| | Creating and deleting snapshot | GA | >= v3.1.0 | >= v1.0.0 | Pacific (>=v16.2.0) | >= v1.17.0 |
|
2024-09-26 08:54:44 +00:00
|
|
|
| | Creating and deleting volume group snapshot | Alpha | >= v3.11.0 | >= v1.9.0 | Squid (>=v19.0.0) | >= v1.31.0 |
|
2023-11-17 09:11:58 +00:00
|
|
|
| | Provision volume from snapshot | GA | >= v3.1.0 | >= v1.0.0 | Pacific (>=v16.2.0) | >= v1.17.0 |
|
|
|
|
|
| | Provision volume from another volume | GA | >= v3.1.0 | >= v1.0.0 | Pacific (>=v16.2.0) | >= v1.16.0 |
|
|
|
|
|
| | Expand volume | Beta | >= v2.0.0 | >= v1.1.0 | Pacific (>=v16.2.0) | >= v1.15.0 |
|
|
|
|
|
| | Volume/PV Metrics of File Mode Volume | GA | >= v1.2.0 | >= v1.1.0 | Pacific (>=v16.2.0) | >= v1.15.0 |
|
|
|
|
|
| NFS | Dynamically provision, de-provision File mode RWO volume | Alpha | >= v3.6.0 | >= v1.0.0 | Pacific (>=v16.2.0) | >= v1.14.0 |
|
|
|
|
|
| | Dynamically provision, de-provision File mode RWX volume | Alpha | >= v3.6.0 | >= v1.0.0 | Pacific (>=v16.2.0) | >= v1.14.0 |
|
|
|
|
|
| | Dynamically provision, de-provision File mode ROX volume | Alpha | >= v3.6.0 | >= v1.0.0 | Pacific (>=v16.2.0) | >= v1.14.0 |
|
|
|
|
|
| | Dynamically provision, de-provision File mode RWOP volume | Alpha | >= v3.6.0 | >= v1.5.0 | Pacific (>=v16.2.0) | >= v1.22.0 |
|
|
|
|
|
| | Expand volume | Alpha | >= v3.7.0 | >= v1.1.0 | Pacific (>=v16.2.0) | >= v1.15.0 |
|
|
|
|
|
| | Creating and deleting snapshot | Alpha | >= v3.7.0 | >= v1.1.0 | Pacific (>=v16.2.0) | >= v1.17.0 |
|
|
|
|
|
| | Provision volume from snapshot | Alpha | >= v3.7.0 | >= v1.1.0 | Pacific (>=v16.2.0) | >= v1.17.0 |
|
|
|
|
|
| | Provision volume from another volume | Alpha | >= v3.7.0 | >= v1.1.0 | Pacific (>=v16.2.0) | >= v1.16.0 |
|
2019-07-12 07:32:44 +00:00
|
|
|
|
|
|
|
|
`NOTE`: The `Alpha` status reflects possible non-backward
|
|
|
|
|
compatible changes in the future, and is thus not recommended
|
|
|
|
|
for production use.
|
2019-05-08 08:18:53 +00:00
|
|
|
|
|
|
|
|
### CSI spec and Kubernetes version compatibility
|
|
|
|
|
|
|
|
|
|
Please refer to the [matrix](https://kubernetes-csi.github.io/docs/#kubernetes-releases)
|
|
|
|
|
in the Kubernetes documentation.
|
|
|
|
|
|
2019-08-14 05:00:15 +00:00
|
|
|
## Ceph CSI Container images and release compatibility
|
|
|
|
|
|
|
|
|
|
| Ceph CSI Release/Branch | Container image name | Image Tag |
|
2019-09-10 11:04:35 +00:00
|
|
|
| ----------------------- | ---------------------------- | --------- |
|
2021-04-16 05:37:03 +00:00
|
|
|
| devel (Branch) | quay.io/cephcsi/cephcsi | canary |
|
2024-12-03 07:09:26 +00:00
|
|
|
| v3.13.0 (Release) | quay.io/cephcsi/cephcsi | v3.13.0 |
|
|
|
|
|
| v3.12.3 (Release) | quay.io/cephcsi/cephcsi | v3.12.3 |
|
2021-08-25 09:37:49 +00:00
|
|
|
|
|
|
|
|
| Deprecated Ceph CSI Release/Branch | Container image name | Image Tag |
|
|
|
|
|
| ----------------------- | --------------------------------| --------- |
|
2024-12-03 07:09:26 +00:00
|
|
|
| v3.11.0 (Release) | quay.io/cephcsi/cephcsi | v3.11.0 |
|
2024-08-14 07:51:56 +00:00
|
|
|
| v3.10.2 (Release) | quay.io/cephcsi/cephcsi | v3.10.2 |
|
|
|
|
|
| v3.10.1 (Release) | quay.io/cephcsi/cephcsi | v3.10.1 |
|
|
|
|
|
| v3.10.0 (Release) | quay.io/cephcsi/cephcsi | v3.10.0 |
|
2024-04-01 09:15:59 +00:00
|
|
|
| v3.9.0 (Release) | quay.io/cephcsi/cephcsi | v3.9.0 |
|
2023-11-27 09:44:19 +00:00
|
|
|
| v3.8.1 (Release) | quay.io/cephcsi/cephcsi | v3.8.1 |
|
|
|
|
|
| v3.8.0 (Release) | quay.io/cephcsi/cephcsi | v3.8.0 |
|
2023-06-26 05:13:15 +00:00
|
|
|
| v3.7.2 (Release) | quay.io/cephcsi/cephcsi | v3.7.2 |
|
|
|
|
|
| v3.7.1 (Release) | quay.io/cephcsi/cephcsi | v3.7.1 |
|
|
|
|
|
| v3.7.0 (Release) | quay.io/cephcsi/cephcsi | v3.7.0 |
|
2023-02-22 07:25:54 +00:00
|
|
|
| v3.6.1 (Release) | quay.io/cephcsi/cephcsi | v3.6.1 |
|
|
|
|
|
| v3.6.0 (Release) | quay.io/cephcsi/cephcsi | v3.6.0 |
|
2022-09-13 13:59:18 +00:00
|
|
|
| v3.5.1 (Release) | quay.io/cephcsi/cephcsi | v3.5.1 |
|
|
|
|
|
| v3.5.0 (Release) | quay.io/cephcsi/cephcsi | v3.5.0 |
|
2022-04-06 08:15:10 +00:00
|
|
|
| v3.4.0 (Release) | quay.io/cephcsi/cephcsi | v3.4.0 |
|
2022-01-17 05:00:08 +00:00
|
|
|
| v3.3.1 (Release) | quay.io/cephcsi/cephcsi | v3.3.1 |
|
|
|
|
|
| v3.3.0 (Release) | quay.io/cephcsi/cephcsi | v3.3.0 |
|
2021-08-25 09:37:49 +00:00
|
|
|
| v3.2.2 (Release) | quay.io/cephcsi/cephcsi | v3.2.2 |
|
|
|
|
|
| v3.2.1 (Release) | quay.io/cephcsi/cephcsi | v3.2.1 |
|
|
|
|
|
| v3.2.0 (Release) | quay.io/cephcsi/cephcsi | v3.2.0 |
|
|
|
|
|
| v3.1.2 (Release) | quay.io/cephcsi/cephcsi | v3.1.2 |
|
|
|
|
|
| v3.1.1 (Release) | quay.io/cephcsi/cephcsi | v3.1.1 |
|
|
|
|
|
| v3.1.0 (Release) | quay.io/cephcsi/cephcsi | v3.1.0 |
|
|
|
|
|
| v3.0.0 (Release) | quay.io/cephcsi/cephcsi | v3.0.0 |
|
|
|
|
|
| v2.1.2 (Release) | quay.io/cephcsi/cephcsi | v2.1.2 |
|
|
|
|
|
| v2.1.1 (Release) | quay.io/cephcsi/cephcsi | v2.1.1 |
|
|
|
|
|
| v2.1.0 (Release) | quay.io/cephcsi/cephcsi | v2.1.0 |
|
|
|
|
|
| v2.0.1 (Release) | quay.io/cephcsi/cephcsi | v2.0.1 |
|
|
|
|
|
| v2.0.0 (Release) | quay.io/cephcsi/cephcsi | v2.0.0 |
|
|
|
|
|
| v1.2.2 (Release) | quay.io/cephcsi/cephcsi | v1.2.2 |
|
|
|
|
|
| v1.2.1 (Release) | quay.io/cephcsi/cephcsi | v1.2.1 |
|
|
|
|
|
| v1.2.0 (Release) | quay.io/cephcsi/cephcsi | v1.2.0 |
|
|
|
|
|
| v1.1.0 (Release) | quay.io/cephcsi/cephcsi | v1.1.0 |
|
|
|
|
|
| v1.0.0 (Branch) | quay.io/cephcsi/cephfsplugin | v1.0.0 |
|
|
|
|
|
| v1.0.0 (Branch) | quay.io/cephcsi/rbdplugin | v1.0.0 |
|
2019-08-14 05:00:15 +00:00
|
|
|
|
2019-04-04 06:43:48 +00:00
|
|
|
## Contributing to this repo
|
|
|
|
|
|
2021-02-25 08:19:23 +00:00
|
|
|
Please follow [development-guide](<https://github.com/ceph/ceph-csi/tree/devel/docs/development-guide.md>)
|
|
|
|
|
and [coding style guidelines](<https://github.com/ceph/ceph-csi/tree/devel/docs/coding.md>)
|
2019-04-04 06:43:48 +00:00
|
|
|
if you are interested to contribute to this repo.
|
|
|
|
|
|
2018-01-20 20:04:38 +00:00
|
|
|
## Troubleshooting
|
|
|
|
|
|
2018-07-31 09:31:12 +00:00
|
|
|
Please submit an issue at: [Issues](https://github.com/ceph/ceph-csi/issues)
|
2019-02-11 14:07:22 +00:00
|
|
|
|
2019-07-25 12:45:07 +00:00
|
|
|
## Weekly Bug Triage call
|
|
|
|
|
|
|
|
|
|
We conduct weekly bug triage calls at our slack channel on Tuesdays.
|
|
|
|
|
More details are available [here](https://github.com/ceph/ceph-csi/issues/463)
|
|
|
|
|
|
2020-04-28 10:18:47 +00:00
|
|
|
## Dev standup
|
|
|
|
|
|
2024-06-25 14:36:21 +00:00
|
|
|
A regular dev standup takes place every [Monday at
|
2023-02-27 08:14:21 +00:00
|
|
|
12:00 PM UTC](https://meet.google.com/vit-qdhw-nyh) Convert to your local
|
2021-04-07 12:30:15 +00:00
|
|
|
timezone by executing command `date -d "12:00 UTC"` on terminal
|
2020-04-28 10:18:47 +00:00
|
|
|
|
|
|
|
|
Any changes to the meeting schedule will be added to the [agenda
|
2022-04-22 11:40:44 +00:00
|
|
|
doc](https://hackmd.io/6GL90WFGQL-L4DcIfIAKeQ).
|
2020-04-28 10:18:47 +00:00
|
|
|
|
|
|
|
|
Anyone who wants to discuss the direction of the project, design and
|
|
|
|
|
implementation reviews, or general questions with the broader community is
|
|
|
|
|
welcome and encouraged to join.
|
|
|
|
|
|
2023-02-27 08:14:21 +00:00
|
|
|
- Meeting link: <https://meet.google.com/vit-qdhw-nyh>
|
2022-04-22 11:40:44 +00:00
|
|
|
- [Current agenda](https://hackmd.io/6GL90WFGQL-L4DcIfIAKeQ)
|
2020-04-28 10:18:47 +00:00
|
|
|
|
2019-05-28 07:13:10 +00:00
|
|
|
## Contact
|
2019-02-11 14:07:22 +00:00
|
|
|
|
2019-05-28 07:13:10 +00:00
|
|
|
Please use the following to reach members of the community:
|
|
|
|
|
|
2024-01-31 15:33:50 +00:00
|
|
|
- Slack: Join the
|
2024-01-31 17:42:44 +00:00
|
|
|
[#ceph-csi](https://ceph-storage.slack.com/archives/C05522L7P60) channel
|
2024-01-31 15:33:50 +00:00
|
|
|
on the [ceph Slack](https://ceph-storage.slack.com) to discuss anything
|
|
|
|
|
related to this project. You can join the Slack by this
|
2024-05-16 11:07:22 +00:00
|
|
|
[invite link](https://bit.ly/ceph-slack-invite)
|
