Commit Graph

290 Commits

Author SHA1 Message Date
Prasanna Kumar Kalever
9a3170bf77 rbd: provide a way to disable the auto fallback to nbd mounter
This change allows the user to choose not to fallback to NBD mounter
when some ImageFeatures are absent with krbd driver, rather just fail
the NodeStage call.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-11-01 08:17:36 +00:00
Niels de Vos
b132696e54 rbd: note that thick-provisioning is deprecated
Thick-provisioning was introduced to make accounting of assigned space
for volumes easier. When thick-provisioned volumes are the only consumer
of the Ceph cluster, this works fine. However, it is unlikely that this
is the case. Instead, accounting of the requested (thin-provisioned)
size of volumes is much more practical as different types of volumes can
be tracked.

OpenShift already provides cluster-wide quotas, which can combine
accounting of requested volumes by grouping different StorageClasses.

In addition to the difficult practise of allowing only thick-provisioned
RBD backed volumes, the performance makes thick-provisioning
troublesome. As volumes need to be completely allocated, data needs to
be written to the volume. This can take a long time, depending on the
size of the volume. Provisioning, cloning and snapshotting becomes very
much noticeable, and because of the additional time consumption, more
prone to failures.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-10-27 06:54:07 +00:00
Robert Vasek
fedbb01ec3 doc: add proposal doc for CephFS snapshots as shallow RO volumes
This patch adds a proposal document for "CephFS snapshots
as shallow RO volumes".

Updates: #2142
Signed-off-by: Robert Vasek <robert.vasek@cern.ch>
2021-10-19 11:35:02 +00:00
Madhu Rajanna
8ebc0659ab rbd: perform resize of file system for static volume
For static volume, the user will manually mounts
already existing image as a volume to the application
pods. As its a rbd Image, if the PVC is of type
fileSystem the image will be mapped, formatted
and mounted on the node,
If the user resizes the image on the ceph cluster.
User cannot not automatically resize the filesystem
created on the rbd image. Even if deletes and
recreates the kubernetes objects, the new size
will not be visible on the node.

With this changes During the NodeStageVolumeRequest
the nodeplugin will check the size of the mapped rbd
image on the node using the devicePath. and also
the rbd image size on the ceph cluster.

If the size is not matching it will do the file
system resize on the node as part of the
NodeStageVolumeRequest RPC call.

The user need to do below operation to see new size
* Resize the rbd image in ceph cluster
* Scale down all the application pods using the static
PVC.
* Make sure no application pods which are using the
static PVC is running on a node.
* Scale up all the application pods.

Validate the new size in application pod mounted
volume.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-10-06 13:15:00 +00:00
Yati Padia
1cf14cd83c cleanup: rework on naming conventions
This commits replaces cephfs -> cephFS
to maintain consistency throughout the
codebase

Updates: #1465

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-09-24 06:17:17 +00:00
Prasanna Kumar Kalever
22bb31df19 doc: update the rbd-nbd doc with log strategies options
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-09-16 13:55:15 +00:00
Prasanna Kumar Kalever
314516cedd deploy: fix cephLogDir passing to storageclass via helm
cephLogDir: is a storage class option that is passed to rbd-nbd daemon.
cephLogDirHostPath: is a nodeplugin daemonset level option that helps in
                   using the right host-path while bind-mounting

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-09-16 13:55:15 +00:00
Madhu Rajanna
43925477a5 doc: update deployment doc for ceph.conf
updated cephfs and rbd deployment doc
to deploy the configmap that contains the
ceph.conf

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-09-03 14:14:43 +00:00
Humble Chirammal
aeebd5d03b doc: remove upgrade instructions for earlier releases
As we have deprecated earlier versions than v3.3.0, it is not required
to keep the upgrade docs for the same. The upgrade doc for v3.2.0 to
v3.3.0 has been kept intact.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-09-01 03:37:23 +00:00
Madhu Rajanna
b0b46680e3 doc: update development guide for new rules
updated development guide requirement to
have review from contributors and reviewers.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-08-25 16:33:56 +05:30
Prasanna Kumar Kalever
7576bf400c doc: update rbd-nbd doc about log path details
Document the changes needed for configuring custom logging path

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-08-24 07:15:30 +00:00
Prasanna Kumar Kalever
396ab1b4d7 doc: update rbd-nbd documentation with encryption volume support details
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-08-11 04:10:34 +00:00
Prasanna Kumar Kalever
85a1fba0f4 doc: update rbd-nbd documentation with volume expansion support details
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-08-10 13:21:16 +00:00
Madhu Rajanna
fce5a181d0 doc: change FsID to FscID for cephfs
updated the filesystem identifier from
FsId to FscID.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-08-05 16:07:51 +00:00
Madhu Rajanna
5fc9c3a046 doc: add design doc for clusterid poolid mapping
added design doc to handle volumeID mapping in case
of the failover in the Disaster Recovery.

update #2118

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-08-03 13:45:58 +00:00
Prasanna Kumar Kalever
d2def71944 doc: update the upgrade documentation to reflect 3.4.0 changes
Mainly removed rbd-nbd mounter specified at the pre-upgrade
considerations affecting the restarts.

Also updated the 3.3 tags to 3.4

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-07-28 11:52:06 +00:00
Prasanna Kumar Kalever
52799da09d doc: add design doc for volume healer
Closes: #667

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-07-28 11:54:59 +05:30
Prasanna Kumar Kalever
068e44bdb1 cleanup: move rbd-mirror image to a new directory
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-07-28 11:54:59 +05:30
Yug Gupta
d14c0afe28 doc: Add documentation for DR
Add documenation for Disaster Recovery
which steps to Failover and Failback in case
of a planned migration or a Disaster.

Signed-off-by: Yug Gupta <yuggupta27@gmail.com>
2021-07-27 11:43:01 +00:00
Niels de Vos
c4372b8567 doc: describe Hashicorp Vault with a ServiceAccount per Tenant
In addition to the single ServiceAccount KMS support for Hashicorp
Vault, Ceph-CSI can now use a ServiceAccount per Tenant as well. This
adds the user-documentation with references to the example deployment
files.

Closes: #2222
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-07-20 12:31:40 +00:00
Niels de Vos
ed298341a6 doc: proposal for KMS with ServiceAccount per Tenant
A new KMS that supports Hashicorp Vault with the Kubernetes Auth backend
and ServiceAccounts per Tenant (Kubernetes Namespace).

Updates: #2222
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-07-13 12:12:25 +00:00
Prasanna Kumar Kalever
8b3136e696 doc: add documentaion for rbd-nbd mounter
Closes #2124

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-07-13 10:19:17 +00:00
Rakshith R
c4060b8aa2 doc: add info about metadata rbd volume encryption
Signed-off-by: Rakshith R <rar@redhat.com>
2021-07-08 17:06:02 +00:00
Humble Chirammal
5dc60126e4 doc: add documentaion about wrapping long lines to coding guide
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-06-28 14:43:49 +00:00
Rakshith R
ba4e92eea9 doc: add imageFeatures parameter info to static-pvc.md
Signed-off-by: Rakshith R <rar@redhat.com>
2021-06-16 06:56:15 +00:00
Yati Padia
cfdf5af08f doc: adds note to replace canary with version tag
This commit adds a note to replace canary with version
tag when deploying in production.

Fixes: #2156

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-06-15 03:14:44 +00:00
Madhu Rajanna
36db988f73 ci: pushing artifacts using github actions
As Travis CI `https://travis-ci.org/` is getting
shutdown date on June 15th. Either we need to move
to new place https://www.travis-ci.com/ or we can
switch to github action to push image and the helm
charts when a PR is merged.

fixes: #1781

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-06-10 10:17:19 +05:30
Humble Chirammal
9aa3520c9d build: update go version to 1.16 in go.mod
Make go version latest in the repo

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-05-25 09:03:52 +00:00
Humble Chirammal
9dc2b1122d doc: correct the keys in certificate secrets
At present the cert keys are not unique which is not correct.
The keys in the secret should be unique and this patch address
the same.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-04-29 08:51:29 +00:00
Madhu Rajanna
029b5004aa doc: update upgrade doc for v3.3.0
As we have v3.3 as the latest release
updating the upgrade doc in the devel
branch to point to the same.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-04-21 06:39:07 +00:00
Humble Chirammal
0fae0e53b6 cleanup: various source code comment corrections
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-04-16 10:22:35 +00:00
Madhu Rajanna
0f8813d89f rbd:store/Read volumeID in/from PV annotation
In the case of the Async DR, the volumeID will
not be the same if the clusterID or the PoolID
is different, With Earlier implementation, it
is expected that the new volumeID mapping is
stored in the rados omap pool. In the case of the
ControllerExpand or the DeleteVolume Request,
the only volumeID will be sent it's not possible
to find the corresponding poolID in the new cluster.

With This Change, it works as below

The csi-rbdplugin-controller will watch for the PV
objects, when there are any PV objects created it
will check the omap already exists, If the omap doesn't
exist it will generate the new volumeID and it checks for
the volumeID mapping entry in the PV annotation, if the
mapping does not exist, it will add the new entry
to the PV annotation.

The cephcsi will check for the PV annotations if the
omap does not exist if the mapping exists in the PV
annotation, it will use the new volumeID for further
operations.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-04-07 11:46:27 +00:00
Humble Chirammal
1b0ebc43d4 deploy: use serviceAccountName instead of serviceAccount in yamls
serviceAccount is the depricated alias for serviceAccountName, so it
is recommended/suggested to use serviceAccountName instead.

For ex. reference:
https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-04-06 09:00:35 +00:00
Niels de Vos
c4856195a2 doc: add description for Amazon KMS provider
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-04-06 07:33:54 +00:00
Humble Chirammal
d80b8d7fc5 doc: correct reference links in the snapshot,release,upgrade docs
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-03-31 06:58:29 +00:00
Madhu Rajanna
d8f7b38d3d rbd: add exclusive-lock and journaling image features for rbd image
Current rbd plugin only supports the layering feature
for rbd image. Add exclusive-lock and journaling image
features for the rbd.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
Signed-off-by: woohhan <woohyung_han@tmax.co.kr>
2021-03-24 09:48:04 +00:00
Prasanna Kumar Kalever
bad7d56ef4 doc: add snapshot API version support matrix
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-03-15 15:11:01 +00:00
Prasanna Kumar Kalever
b9291c74f0 doc: adjust SNAPSHOT_VERSION
Fixes: #1803
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-03-15 15:11:01 +00:00
Niels de Vos
22ee7c0de5 doc: add implementation details
Update the emcrypted PVC implementation doc with references to the new
EncryptedKMS, DEKStore and VolumeEncryption types.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-03-12 10:11:47 +00:00
Niels de Vos
06d5d8f23a build: libcephfs-devel is not needed
go-ceph does not  use CephFS development headers, so there is no need to
install libcephfs-devel.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-03-10 09:26:00 +00:00
Niels de Vos
253858c104 doc: rename "master" branch to "devel"
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-03-01 10:51:30 +05:30
Madhu Rajanna
c417a5d0ba rbd: add support for thick provisioning option
Add an option to the StorageClass to support creating fully allocated
(thick provisioned) RBD images

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-02-19 11:55:40 +00:00
Niels de Vos
4937e59c4d rbd: add backwards compatible encryption in NodeStageVolume
When a volume was provisioned by an old Ceph-CSI provisioner, the
metadata of the RBD image will contain `requiresEncryption` to indicate
a passphrase needs to be created. New Ceph-CSI provisioners create the
passphrase in the CreateVolume request, and set `encryptionPrepared`
instead.

When a new node-plugin detects that `requiresEncryption` is set in the
RBD image metadata, it will fallback to the old behaviour.

In case `encryptionPrepared` is read from the RBD image metadata, the
passphrase is used to cryptsetup/format the image.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-02-17 17:51:13 +00:00
Madhu Rajanna
22ae4a0b16 rbd: change key in secret for cert and tls
currently, the keys for kms certificates/keys in a
secret is ca.cert, tls.cert and
tls.key, this commit changes the key from ca.cert
and tls.cert to cert and tls.key to key.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-02-04 14:58:40 +00:00
Flemming Frandsen
47e12a6b6c doc: explain why certain features are unsupported
Signed-off-by: Flemming Frandsen <dren.dk@gmail.com>
2021-01-15 13:10:38 +00:00
Madhu Rajanna
81061e9f68 util: add support for vault certificates
Added a option to pass the client certificate
and the client certificate key for the vault token
based encryption.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-16 11:01:15 +00:00
Niels de Vos
b8fec4df64 doc: fix links for example yaml files
The yaml files for RBD encryption are located in examples/kms/vault, and
not in the examples/rbd directory.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-14 11:14:50 +00:00
Niels de Vos
db40c06e84 doc: add usage for Vault Tokens KMS support
In addition to the Vault KMS support (uses Kubernetes ServiceAccount),
there is the new Vault Tokens KMS feature.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-14 11:14:50 +00:00
Niels de Vos
24a17094a2 doc: multi-tenancy with Vault tokens
Design for adding a new KMS type "VaultTokens" that can be used to
configure a Hashicorp Vault service where each tenant has their own
personal token to manage encryptions keys for PVCs.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-10 15:39:47 +00:00
Mudit Agarwal
ba8ead6e04 doc: add upgrade document for 3.2
Added upgrade document for 3.2 with a separate section on
CSI sidecar containers update.

Signed-off-by: Mudit Agarwal <muagarwa@redhat.com>
2020-12-07 15:16:46 +00:00
yati1998
78968f76e8 doc: update snap-clone with rbd snapshot
RBD Snapshot doc was the part of the README.md file. Hence,
renamed the cephfs-snap-clone.md file to snap-clone.md file
and moved the rbd snapshot document there.

Signed-off-by: yati1998 <ypadia@redhat.com>
2020-12-02 21:48:39 +00:00
Madhu Rajanna
8ebb9a1ba0 cleanup: fix misspell words
fixed misspell words detected by  codespell

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-11-30 08:46:48 +01:00
Madhu Rajanna
28793efc90 doc: design document for rbd mirroring
This document outlines the internal cephcsi
design to handle mirrored RBD images.

Co-authored-by: ShyamsundarR <srangana@redhat.com>
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-11-29 07:49:11 +00:00
Madhu Rajanna
39b1f2b4d3 cleanup: fix mispell words
fixed mispell words in the repo.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-11-29 12:47:46 +05:30
Prasanna Kumar Kalever
817edfd1c7 cleanup: remove the use of text in markdown
We do not have `text` in the new section of the MarkDown Rules. Hence
dropping them.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2020-11-11 13:18:05 +00:00
Prasanna Kumar Kalever
8475a3b97e doc: update about a markdown rule in coding guide
Update the coding guide about MD014, i.e.
Dollar signs used before commands without showing output

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2020-11-11 13:18:05 +00:00
Prasanna Kumar Kalever
2945f7b669 cleanup: stick to standards when using dollar-sign in md
MD014 - Dollar signs used before commands without showing output
The dollar signs are unnecessary, it is easier to copy and paste and
less noisy if the dollar signs are omitted. Especially when the
command doesn't list the output, but if the command follows output
we can use `$ ` (dollar+space) mainly to differentiate between
command and its ouput.

scenario 1: when command doesn't follow output
```console
cd ~/work
```

scenario 2: when command follow output (use dollar+space)
```console
$ ls ~/work
file1 file2 dir1 dir2 ...
```

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2020-11-11 13:18:05 +00:00
Satoru Takeuchi
79e7c6a3e2 doc: remove the description of provisioner statefulset
Provisioners don't use StatefulSet anymore.

Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
2020-10-23 06:48:33 +00:00
Prasanna Kumar Kalever
ea5264220e doc: update developer guide about retriggering CI jobs
Add instructions about how and when to retrigger the CI jobs

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2020-10-21 22:21:57 +00:00
Prasanna Kumar Kalever
da056a5ef6 doc: add install-snapshot.sh usage in snapshot documentation
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2020-10-14 13:17:04 +00:00
Humble Chirammal
36ee8b2240 rbd: make sure csi rbd pv encryption defaulting to luks2 version
Fixes: #https://github.com/ceph/ceph-csi/issues/1564

As per release notes this new version is compatible with previous
version of LUKS.
https://www.saout.de/pipermail/dm-crypt/2017-December/005771.html

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-10-13 05:11:25 +00:00
Humble Chirammal
0b5da6a127 doc: update encryption doc for rbd
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-10-08 11:24:26 +00:00
Prasanna Kumar Kalever
e03ea1dc3a doc: parameter imageFeatures is required
fix imageFeatures parameter in the deploy-rbd doc

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2020-10-07 13:01:45 +00:00
Jeremy Facchetti
b9dd54e238 doc: added precisions about snapshotting and cloning
Signed-off-by: Jeremy Facchetti <facchettos@gmail.com>
2020-09-29 12:48:30 +00:00
Madhu Rajanna
43f28af495 doc: replace nautilus with master in ceph doc link
refer to master branch of ceph docs instead of nautilus.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-09-21 13:27:28 +00:00
Madhu Rajanna
d1f175d9f3 rbd: add support for rbd map and unmap options
added support for providing map and unmap
options to rbd CLI when mapping rbd image
on the node.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-09-21 13:27:28 +00:00
Niels de Vos
200d8089f5 doc: add description of most useful GitHub labels
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-08-25 11:18:25 +00:00
Yug
843ea1ce31 doc: Correct cli arguments matrix
Correct the matrix for deploy-rbd document.

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-08-24 07:12:42 +00:00
Yug
2d050e3ff0 doc: correct the cli arguments matrix
The current matrix seems to be showing
the undesired values which is corrected
with this commit.

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-08-24 07:12:42 +00:00
Yug
c73fa55b97 doc: Add step to verify if PVC is in Bound State
Verify if PVC is in Bound state before it's
snapshot is created.

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-08-24 07:12:42 +00:00
Yug
d9c8b07b84 doc: Correct title of capabilities doc
Correct capabilities doc with minor nits.

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-08-24 07:12:42 +00:00
Yug
e3fb20de21 doc: Correct minor nits
Correct development-guide doc with minor
nit picks.

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-08-24 07:12:42 +00:00
Yug
f74922fe57 doc: Correct static-pvc doc
Correct minor nit in static-pvc doc.

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-08-24 07:12:42 +00:00
Yug
28f02cefa2 doc: Add missing keyword in static-pvc doc
The keyword `clusterID` was missing in a
couple sentences; which can be added for
better understanding.

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-08-20 08:36:36 +00:00
Yug
78c1e781be doc: Add steps to set up git hook
Provide steps to install and set up
git hook locally.

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-08-19 16:01:16 +00:00
yati1998
75d800c07e doc: documentation updates on capabilities
Documents exact capabilities a id requires against a
Ceph cluster for RBD and CephFS plugins.

Signed-off-by: yati1998 <ypadia@redhat.com>
2020-08-17 09:47:03 +00:00
Madhu Rajanna
f11486f4b6 doc: add note for cephfs snapshot limitation
added a note for cephfs snapshot limitations

closes #1242

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 11:55:23 +00:00
Madhu Rajanna
fabe84d4fe doc: add upgrade documentation for v3.1.0
added upgrade documentation to upgrade from
v3.0.0 to v3.1.0

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 11:03:36 +00:00
Humble Chirammal
3c6149b86e cephfs: Add documentation about snapshot create/restore and clone
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-13 07:16:31 +00:00
Humble Chirammal
07890aace6 doc: correct rbd snap-clone doc
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-11 13:39:47 +00:00
Madhu Rajanna
4937ee97e9 doc: correct upgrade doc
fixed the missing `v` version in upgrade
doc.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-27 08:10:41 +00:00
Madhu Rajanna
a21d8fad69 doc: update upgrade doc for v3.0.0
updated upgrade documentation for upgrade
from v2.1.x to v3.0 .

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-27 11:29:20 +05:30
Madhu Rajanna
b18fca7ae0 doc: Remove support for mimic
As ceph mimic is deprecated in the ceph upstream,
we are removing the support for mimic from ceph-csi
also, the user need to update the latest Nautilus or
Octopus to use ceph-csi.

more info realated to ceph mimim deprecation at
https://lists.ceph.io/hyperkitty/list/dev@ceph.io/thread/X5IUICDEM4IVVWTMUTSSNEU424MB6WL7/
https://ceph.io/releases/mimic-is-retired/

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-23 04:44:37 +00:00
Madhu Rajanna
cf98442ef6 doc: add document for rbd snapshot and clone
Added a document which contains the steps
and RBD CLI commands we execute when we create
a kubernetes snapshot, delete kubernetes snapshot,
Restore a snapshot to a new PVC,Kubernetes volume
cloning and kubernetes PVC deletion.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-20 12:56:23 +00:00
Madhu Rajanna
d15ded88f5 cleanup: Remove support for Delete and Unmounting v1.1.0 PVC
as v1.0.0 is deprecated we need to remove the support
for it in the Next coming (v3.0.0) release. This PR
removes the support for the same.

closes #882

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-10 16:07:13 +00:00
Madhu Rajanna
8ef7143e6c rbd: add maxsnapshotsonimage flag
Added maxsnapshotsonimage flag to flatten
the older rbd images on the chain to avoid
issue in krbd.The limit is in krbd since it
only allocate 1 4KiB page to handle all the
snapshot ids for an image.

The max limit is 510 as per
https://github.com/torvalds/linux/blob/
aaa2faab4ed8e5fe0111e04d6e168c028fe2987f/drivers/block/rbd.c#L98
in cephcsi we arekeeping the default to 450 to reserve 10%
to avoid issues.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-06 10:08:31 +00:00
Madhu Rajanna
826f7126cd doc: update doc to delete alpha snapshot
updated upgrade documentation to remove
the snapshot created by alpha driver before
upgrade of CSI driver as beta snapshot is not
backward compatible with the alpha snapshot.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-03 11:32:00 +00:00
Niels de Vos
e393e298d0 doc: do not mention "github.com/pkg/errors"
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-03 09:12:48 +00:00
Madhu Rajanna
b085577a4f rbd: add skipForceFlatten flag
added skipForceFlatten flag to skip
the image deptha and skip image flattening.
This will be very useful if the kernel is
not listed in cephcsi which supports deep
flatten fauture.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-01 08:21:47 +00:00
Madhu Rajanna
9b518726ab rbd: add hardlimt and softlimit flag
added Hardlimit and Softlimit flags for cephcsi
arguments. When the Softlimit is reached cephcsi
will start a background task to flatten the rbd
image and return success and if the hardlimit
is reached it will start a background task
to flatten the rbd image and return ready
to use as false to make sure that the image
will not be used until it is flatten.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-01 08:21:47 +00:00
Madhu Rajanna
1a1ad11f57 doc: update coding doc to correct import order
Updated coding doc to correct the import order
as per the standard. More info can be found on
https://github.com/golang/go/wiki/CodeReviewComments#imports

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-06-22 09:51:23 +00:00
Humble Chirammal
134e11e26e doc: correct PVC expansion documentation
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-06-16 10:31:29 +00:00
Humble Chirammal
2793b79ad8 doc: Correct PVC mention in the deployment doc
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-06-11 10:53:10 +00:00
Humble Chirammal
c1a269191c doc: make sure configmap object referred or documented correctly
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-06-10 11:33:10 +00:00
Yug Gupta
6cf7389d97 doc: Add prefix for naming subvolumes and snapshots
Updated storageclass and snapshotclass
to include the name prefix for naming
subvolumes and snapshots.

Fixes: #1087

Signed-off-by: Yug Gupta <ygupta@redhat.com>
2020-05-27 05:54:10 +00:00
Madhu Rajanna
78267692fc CI: Add Rule to auto merge PR
If the PR is having trivial changes or the reviewer is
confident enough that PR doesn't need a second review,
the reviewer can set `ready-to-merge` label on the PR.
The bot will merge the PR if it's having one approval and the
label `ready-to-merge`

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-05-20 14:27:33 +05:30
Mudit Agarwal
cae8f529dc docs: add doc for cephFS static PV
With the change in #382, support for static PV for CephFS was added.
This change is to update the already existing doc for the same.

Issue: #669

Signed-off-by: Mudit Agarwal <muagarwa@redhat.com>
2020-05-18 08:30:50 +00:00
Madhu Rajanna
1a3b07994e doc: Add Upgrade documentation for snapshot
Added step to identify alpha snapshot CRD.
Added step to delete alpha CRD and link
for installing beta CRD.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-05-15 08:19:32 +00:00
Niels de Vos
25ea25368a doc: add list of valid components to the development guide
The commitlint CI job uses the configuration from .commitlintrc.yaml
which contains the different components that Ceph-CSI uses. A short
description of each component has been added, so that contributors
understand what component to mention in the prefix of the subject in
commit messages.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-05-14 08:45:26 +00:00
Madhu Rajanna
9ec4479234 DOC: Added document for DOC and commit message
Added document on the standard user need to follow
when writting the commit message and to include
sign-off in commit message.

source: https://probot.github.io/apps/dco/

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-05-06 16:54:15 +00:00
Madhu Rajanna
919f3b6d85 Doc: update development documentation
Updated golang version to 1.13.x and
also updated user to set GO111MODULE=on
and CGO_ENABLED=1 when doing development
in cephcsi

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-05-05 07:56:38 +00:00
Niels de Vos
32839948ef cleanup: move pkg/ to internal/
The internal/ directory in Go has a special meaning, and indicates that
those packages are not meant for external consumption. Ceph-CSI does
provide public APIs for other projects to consume. There is no plan to
keep the API of the internally used packages stable.

Closes: #903
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-04-23 11:00:59 +00:00
Niels de Vos
718e76d290 doc: add description for containerized-test to development-guide
Suggested-by: Wilmar den Ouden <info@wilmardenouden.nl>
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-04-22 10:50:18 +00:00
Niels de Vos
7bdb42c14a doc: update to current release tracking method
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-04-17 09:16:58 +00:00
Madhu Rajanna
2099600aa6 Upgrade: Upgrade doc for v2.1.0
This PR updates the upgrade document
from v2.0.x to v2.1.0

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-04-15 09:48:31 +00:00
ShyamsundarR
5c4abf8347 Add topology support to ceph-csi
Signed-off-by: ShyamsundarR <srangana@redhat.com>
2020-04-14 14:14:29 +00:00
chenxu1990
5475022bc3 Document about stale resource cleanup
1. when user delete pv manual, it will result in stale metadata and image in ceph
2020-04-14 11:29:07 +00:00
Niels de Vos
a1de56dbd3 tests: in case 'go test' is run in a container, skip TestGetPIDLimit()
In (standard, non-privileged) container environments the /sys/fs/cgroup
mountpoint is not available. This would cause the tests to fail, as
TestGetPIDLimit() tries to write to the cgroup configuration.

The test will work when run as root on a privileged container or
directly on a host (as Travis CI does).

Setting the CEPH_CSI_RUN_ALL_TESTS environment variable to a non-empty
value will cause the test to be executed.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-04-02 06:08:03 +00:00
Madhu Rajanna
bcd646ee55 Deprecate grpc metrics in ceph-csi
As kubernetes CSI sidecar is exposing the
GRPC mertics we can make use of the same in
ceph-csi we dont need to expose our own.

update: #881

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-04-01 11:59:37 +00:00
Niels de Vos
7381253ee0 build: add an option to compile in a container
This makes it possible to build on any platform that supports Linux
containers. The container image used for building is created once, or on
updating the `scripts/Dockerfile.build` and is cached afterwards.

To build the executable in a container, use `make containerized-build`
and everything will be done automatically. The executable will also be
available on the usual location.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-03-26 08:45:53 +00:00
Niels de Vos
d15b77d403 dev-guide: add reference to required go-ceph dependencies
Closes: #872
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-03-19 17:50:49 +00:00
Madhu Rajanna
7f8c535c42 Update upgrade doc for node hang issue
This PR updates the upgrade doc to handle the
node drain issue what we have seen in
https://github.com/ceph/ceph-csi/issues/756

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-03-19 14:56:04 +00:00
Reinier Schoof
a4532fafd0 added volumeNamePrefix and snapshotNamePrefix as parameters for storageClass
this allows administrators to override the naming prefix for both volumes and snapshots
created by the rbd plugin.

Signed-off-by: Reinier Schoof <reinier@skoef.nl>
2020-02-25 05:03:51 +00:00
Madhu Rajanna
8163552b81 Add doc for rbd static pvc support
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-02-18 12:34:46 +00:00
Madhu Rajanna
034b123478 Remove mount cache for cephfs
PR #282 introduces the mount cache to
solve cephfs fuse mount issue when cephfs plugin pod
restarts .This is not working as intended. This PR removes
the code for maintainability.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-02-11 15:11:21 +00:00
Vasyl Purchel
669dc4536f Reduce encryption KMS configuration SC parameters
* moves KMS type from StorageClass into KMS configuration itself
 * updates omapval used to identify KMS to only it's ID without the type

why?

1. when using multiple KMS configurations (not currently supported)
automated parsing of kms configuration will be failing because some
entries in configs won't comply with the requested type
2. less options are needed in the StorageClass and less data used to
identify the KMS

Signed-off-by: Vasyl Purchel vasyl.purchel@workday.com
Signed-off-by: Andrea Baglioni andrea.baglioni@workday.com
2020-02-10 15:21:11 +00:00
Vasyl Purchel
419ad0dd8e Adds per volume encryption with Vault integration
- adds proposal document for PVC encryption from PR448
- adds per-volume encription by generating encryption passphrase
  for each volume and storing it in a KMS
- adds HashiCorp Vault integration as a KMS for encryption passphrases
- avoids encrypting volume second time if it was already encrypted but
  no file system created
- avoids unnecessary checks if volume is a mapped device when encryption
  was not requested
- prevents resizing encrypted volumes (it is not currently supported)
- prevents creating snapshots from encrypted volumes to prevent attack
  on encryption key (security guard until re-encryption of volumes
  implemented)

Signed-off-by: Vasyl Purchel vasyl.purchel@workday.com
Signed-off-by: Andrea Baglioni andrea.baglioni@workday.com

Fixes #420
Fixes #744
2020-02-05 05:18:56 +00:00
Oguz Kilcan
aadce54b2f Added PodSecurityPolicy support 2020-01-22 08:19:42 +00:00
wilmardo
f04af5742d refact: Remove Kubernetes 1.13.x support
Signed-off-by: wilmardo <info@wilmardenouden.nl>
2020-01-20 10:32:30 +00:00
Madhu Rajanna
369fdb5051 Doc for cephfs expand PVC
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-01-17 13:45:20 +00:00
Madhu Rajanna
6a8ddad669 Add Ceph-csi Upgrade documentation
Added Upgrade doc from v1.2.x to v2.0.0

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-01-14 07:50:52 +00:00
Humble Chirammal
866bc7da21 Add resize doc for both filesystem and block mode PVCs
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-01-13 14:53:08 +00:00
Vasyl Purchel
166eaf700f Adds PVC encryption with LUKS
Adds encryption in StorageClass as a parameter. Encryption passphrase is
stored in kubernetes secrets per StorageClass. Implements rbd volume
encryption relying on dm-crypt and cryptsetup using LUKS extension

The change is related to proposal made earlier. This is a first part of
the full feature that adds encryption with passphrase stored in secrets.

Signed-off-by: Vasyl Purchel vasyl.purchel@workday.com
Signed-off-by: Andrea Baglioni andrea.baglioni@workday.com
Signed-off-by: Ioannis Papaioannou ioannis.papaioannou@workday.com
Signed-off-by: Paul Mc Auley paul.mcauley@workday.com
Signed-off-by: Sergio de Carvalho sergio.carvalho@workday.com
2019-12-16 08:12:44 +00:00
Madhu Rajanna
118f34525e Remove deprecated containerized
As we are moving towards v2.0.0 I think
it's a good time to remove the deprecated flag.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-12-11 14:44:35 +00:00
Stefan Haas
6a2717ce20 Added forcecephkernelclient as startup parameter to force enabling ceph
Signed-off-by: Stefan Haas <shaas@suse.com>
2019-10-16 06:47:10 +00:00
Madhu Rajanna
136d81b736 Add mergify rule to auto merge backported PR
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-09-27 16:11:33 +05:30
wilmardo
6ee381db3a refactor: Merge 1.13 and 1.14 Helm charts and improve charts
Signed-off-by: wilmardo <info@wilmardenouden.nl>
2019-09-27 05:49:18 +00:00
Madhu Rajanna
ceef9ff3a5 Backport PR when label is set
backport PR to release-v1.2.0 branch
from master branch  when label is set
on PR

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-09-27 09:50:59 +05:30
Madhu Rajanna
70d49b4e47 tempate changes for containerized flag removal
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-09-23 13:22:29 +00:00
Madhu Rajanna
6da96c6327 remove support for create image with image-format 1
tried to create an image with image-format=1

```
sh-4.2# rbd create --size=1024 replicapool/test --image-format=1
rbd: image format 1 is deprecated
rbd: create error: (22) Invalid argument
2019-09-11 07:00:54.531 7fb0e40bfb00 -1 librbd: Format 1 image creation unsupported.
```

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-09-12 07:52:32 +00:00
Madhu Rajanna
41b701c98c Add support for erasure pool in rbd
Allow specifying different metadata and data pools in a
CSI RBD StorageClass

Fixes: #199
Fixes: https://github.com/rook/rook/issues/2650
Fixes: https://github.com/rook/rook/issues/3763

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-09-11 06:48:08 +00:00
Poornima G
060ff8d25e Add mount option for Cephfs
The storage class already takes MountOptions(MountFlags), these are the
bind mount options. Some of these options may not be recognised by the
cephfs mount. Hence added a new parameterin Storage Class for
- cephfs kernel mount options,
- ceph-fuse mount options

Ceph kernel mount options are different from ceph-fuse options, hence
added two different parameters.

Signed-off-by: Poornima G <pgurusid@redhat.com>
2019-09-06 16:32:10 +00:00
Madhu Rajanna
f4b38228ae Remove volumemounter flag from cephfs
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-09-05 07:20:50 +00:00
Madhu Rajanna
a81a3bf96b implement grpc metrics for ceph-csi
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-08-30 06:50:32 +00:00
wilmardo
30fb7de118 feat: Implement helm lint
Signed-off-by: wilmardo <info@wilmardenouden.nl>
2019-08-16 07:38:33 +00:00
Daniel-Pivonka
d621a58207 prometheus liveness probe sidecar
Signed-off-by: Daniel-Pivonka dpivonka@redhat.com
2019-08-13 17:51:41 +00:00
wilmardo
c739ce9d5e Removes last reference to node-publish-secret
Signed-off-by: wilmardo <info@wilmardenouden.nl>
2019-08-13 16:42:15 +00:00
Niels de Vos
31648c8feb provisioners: add reconfiguring of PID limit
The container runtime CRI-O limits the number of PIDs to 1024 by
default. When many PVCs are requested at the same time, it is possible
for the provisioner to start too many threads (or go routines) and
executing 'rbd' commands can start to fail. In case a go routine can not
get started, the process panics.

The PID limit can be changed by passing an argument to kubelet, but this
will affect all pids running on a host. Changing the parameters to
kubelet is also not a very elegant solution.

Instead, the provisioner pod can change the configuration itself. The
pod is running in privileged mode and can write to /sys/fs/cgroup where
the limit is configured.

With this change, the limit is configured to 'max', just as if there is
no limit at all. The logs of the csi-rbdplugin in the provisioner pod
will reflect the change it makes when starting the service:

    $ oc -n rook-ceph logs -c csi-rbdplugin csi-rbdplugin-provisioner-0
    ..
    I0726 13:59:19.737678       1 cephcsi.go:127] Initial PID limit is set to 1024
    I0726 13:59:19.737746       1 cephcsi.go:136] Reconfigured PID limit to -1 (max)
    ..

It is possible to pass a different limit on the commandline of the
cephcsi executable. The following flag has been added:

    --pidlimit=<int>       the PID limit to configure through cgroups

This accepts special values -1 (max) and 0 (default, do not
reconfigure). Other integers will be the limit that gets configured in
cgroups.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2019-08-13 14:43:29 +00:00
ShyamsundarR
44f7b1fe4b Use "rbd device list" to list and find rbd images and their device paths
This change also starts mapping nbd based access using ther rbd CLI
as, it is a prerequisite to get device listing for nbd as well.

Signed-off-by: ShyamsundarR <srangana@redhat.com>
2019-08-13 14:07:52 +00:00
William Zhang
44e807c36b Add the description of Deploy ConfigMap for CSI plugins
Signed-off-by: William Zhang <zhang.wanmin@zte.com.cn>
2019-08-08 12:53:31 +00:00
Daniel-Pivonka
0063727199 Make parameter pool optional in CephFS storageclass
Signed-off-by: Daniel-Pivonka <dpivonka@redhat.com>
2019-08-07 13:30:38 +00:00
Madhu Rajanna
02bcb5f16a Enable leader election in v1.14+
Use Deployment with leader election instead of StatefulSet

Deployment behaves better when a node gets disconnected
from the rest of the cluster - new provisioner leader
is elected in ~15 seconds, while it may take up to
5 minutes for StatefulSet to start a new replica.

Refer: kubernetes-csi/external-provisioner@52d1fbc

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-08-05 07:11:44 +00:00
Humble Devassy Chirammal
c7d990a96b
Merge pull request #460 from Madhu-1/fix-pluginapath
Fix pluginpath for cephfs
2019-07-29 14:02:18 +05:30
Ramana Raja
7dc723ebbc docs: add special config steps to make CephFS CSI v1.1.0
... work with Ceph v14.2.2 cluster that is not deployed by rook.

Fixes: #486
Signed-off-by: Ramana Raja <rraja@redhat.com>
2019-07-26 11:36:09 +00:00
Madhu Rajanna
778cfb3090 provide option to set pluginpath for cephfs
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-07-25 14:47:42 +05:30
ShyamsundarR
fa68c35f3b Support mounting and deleting version 1.0.0 RBD volumes
This commit adds support to mount and delete volumes provisioned by older
plugin versions (1.0.0) in order to support backward compatibility to 1.0.0
created volumes.

It adds back the ability to specify where older meta data was specified, using
the metadatastorage option to the plugin. Further, using the provided meta data
to mount and delete the older volumes.

It also supports a variety of ways in which monitor information may have been
specified (in the storage class, or in the secret), to keep the monitor
information current.

Testing done:
- Mount/Delete 1.0.0 plugin created volume with monitors in the StorageClass
- Mount/Delete 1.0.0 plugin created volume with monitors in the secret with
  a key "monitors"
- Mount/Delete 1.0.0 plugin created volume with monitors in the secret with
  a user specified key
- PVC creation and deletion with the current version (to ensure at the minimum
  no broken functionality)
- Tested some negative cases, where monitor information is missing in secrets
  or present with a different key name, to understand if failure scenarios work
  as expected

Updates #378

Follow-up work:
- Documentation on how to upgrade to 1.1 plugin and retain above functionality
  for older volumes

Signed-off-by: ShyamsundarR <srangana@redhat.com>
2019-07-08 15:40:17 +00:00
ShyamsundarR
c5762b6b5c Modify RBD plugin to use a single ID and move the id and key into the secret
RBD plugin needs only a single ID to manage images and operations against a
pool, mentioned in the storage class. The current scheme of 2 IDs is hence not
needed and removed in this commit.

Further, unlike CephFS plugin, the RBD plugin splits the user id and the key
into the storage class and the secret respectively. Also the parameter name
for the key in the secret is noted in the storageclass making it a variant and
hampers usability/comprehension. This is also fixed by moving the id and the key
to the secret and not retaining the same in the storage class, like CephFS.

Fixes #270

Testing done:
- Basic PVC creation and mounting

Signed-off-by: ShyamsundarR <srangana@redhat.com>
2019-06-24 13:46:14 +00:00
Humble Devassy Chirammal
95252dd9f6
Merge pull request #390 from ShyamsundarR/stateless-cephfs
Make CephFS plugin stateless reusing RADOS based journal scheme
2019-06-07 10:44:18 +05:30
Humble Chirammal
69c4e7537f Update coding.md
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2019-06-04 11:50:52 +05:30
Humble Devassy Chirammal
1444231d05
Merge pull request #304 from humblec/contributing
Add coding.md into docs
2019-06-04 11:40:47 +05:30