Commit Graph

79 Commits

Author SHA1 Message Date
Rakshith R
40de75e0db rbd: modify oidc token file path according to FHS 3.0
OIDC token file path has been modified from
`/var/run/secrets/token` to `/run/secrets/tokens`.
This has been done to ensure compliance with
FHS 3.0.

refer:
https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch05s13.html

Signed-off-by: Rakshith R <rar@redhat.com>
2022-03-23 13:29:35 +00:00
Rakshith R
4f0bb2315b rbd: add aws-sts-metdata encryption type
With Amazon STS and kubernetes cluster is configured with
OIDC identity provider, credentials to access Amazon KMS
can be fetched using oidc-token(serviceaccount token).
Each tenant/namespace needs to create a secret with aws region,
role and CMK ARN.
Ceph-CSI will assume the given role with oidc token and access
aws KMS, with given CMK to encrypt/decrypt DEK which will stored
in the image metdata.

Refer: https://docs.aws.amazon.com/STS/latest/APIReference/welcome.html
Resolves: #2879

Signed-off-by: Rakshith R <rar@redhat.com>
2022-03-16 07:29:56 +00:00
Silvio Gissi
9c50e255fb helm: make ceph.conf ConfigMap name configurable
ConfigMap name was hardcoded and led to conflicts. Fixes #2858.

Signed-off-by: Silvio Gissi <silvio@gissilabs.com>
2022-02-21 07:25:22 +00:00
Francesco Astegiano
4235178f7c helm: Add selinuxMount flag to enable/disable /etc/selinux host mount
Add selinuxMount flag to enable/disable /etc/selinux host mount inside pods
to support selinux-enabled filesystems

Signed-off-by: Francesco Astegiano <francesco.astegiano@gmail.com>
2022-02-16 12:48:00 +00:00
Deividas Burškaitis
91c22f521b helm: add port sections to helm templates
to show what ports containers are exposing add port sections to nodeplugin
and provisioner helm templates

Signed-off-by: Deividas Burškaitis <deividas.burskaitis@oxylabs.io>
2022-02-15 10:06:26 +00:00
Madhu Rajanna
0311eb5f44 helm: remove namespace from storageclass yaml
removes namespace from non-namespaced storageclass
object.

fixes: #2714

Replacement for #2715 as we didnt receive any update
and PR is already closed.

Co-authored-by: jhrcz-ls
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-02-11 12:32:58 +00:00
Humble Chirammal
de2489ed7d deploy: update csi-snapshotter sidecar to v5.0.1
This release of snapshotter has a breaking change as mentioned
in the release note:

Refer#
[1]: https://github.com/kubernetes-csi/external-snapshotter/releases/tag/v5.0.0

RBAC rules are also updated with this commit.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-02-03 19:01:57 +00:00
Madhu Rajanna
28fef9b379 cleanup: remove thick provisioning code
This commit removes the thick provisioning
code as thick provisioning is deprecated in
cephcsi 3.5.0.

fixes: #2795

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-01-28 11:17:15 +00:00
Niels de Vos
ee2e97b62d deploy: add CSI-Addons endpoint
Deployments place all sockets for communicating with CSI components in
the shared `/csi` directory. The CSI-Addons socket was introduced
recently, but not configured to be in the same location (by default
placed in `/tmp`).

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-12-22 13:21:59 +00:00
Steven Reitsma
6be0e8cb51 helm: Fix missing ClusterRoleBinding for nodeplugin ServiceAccount
When topology is disabled, the ClusterRoleBinding is not created in the Helm
chart. However, the nodeplugin needs access to volumeattachments for the volume
healer.

Signed-off-by: Steven Reitsma <steven@properchaos.nl>
2021-12-22 11:06:11 +00:00
Toby Jackson
989905aa9f helm: csiplugin-configmap generates invalid configuation
When generating csiconfiguration from values the config.json key gets merged with cluster-mapping.json
as the config.json toYaml element supresses a newline.

This fixes the situation where configuration is generated as shown;

```
 data:
   config.json: |-
    [{"clusterID":"....","monitors":["..."]}]cluster-mapping.json: |-
    []
```

Signed-off-by: Toby Jackson <toby@warmfusion.co.uk>
2021-11-25 06:31:38 +00:00
Prasanna Kumar Kalever
9a3170bf77 rbd: provide a way to disable the auto fallback to nbd mounter
This change allows the user to choose not to fallback to NBD mounter
when some ImageFeatures are absent with krbd driver, rather just fail
the NodeStage call.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-11-01 08:17:36 +00:00
Shaohui Liu
af752dd38f helm: support adding annotations to StorageClasses
Signed-off-by: Shaohui Liu <liushaohui@xiaomi.com>
2021-10-28 16:56:12 +00:00
Madhu Rajanna
e45bf03bd8 helm: remove securityContext from cephfs deployment
we dont need securityContext for the cephfs provisioner
pod as its not doing any special operations like mount,
selinux operations etc .

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-09-22 07:12:34 +00:00
Madhu Rajanna
f267d77801 helm: remove extra volumes from rbd plugin PSP
removed extra volume permissions from the rbd
nodeplugin PSP

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-09-22 07:12:34 +00:00
Madhu Rajanna
6f1066fd36 helm: reduce the PSP permission for rbd deployment
rbd deployment doesnot need extra permission like
privileged and extra volumes etc.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-09-22 07:12:34 +00:00
Prasanna Kumar Kalever
c9cc36d8db rbd: provide alternatives to preserve the ceph log files
Currently, we delete the ceph client log file on unmap/detach.

This patch provides additional alternatives for users who would like to
persist the log files.

Strategies:
-----------
`remove`: delete log file on unmap/detach
`compress`: compress the log file to gzip on unmap/detach
`preserve`: preserve the log file in text format

Note that the default strategy will be remove on unmap, and these options
can be tweaked from the storage class

Compression size details example:

On Map: (with debug-rbd=20)
---------
$ ls -lh
-rw-r--r-- 1 root root 526K Sep  1 18:15
rbd-nbd-0001-0024-fed5480a-f00f-417a-a51d-31d8a8144c03-0000000000000003-d2e89c87-0b4d-11ec-8ea6-160f128e682d.log

On unmap:
---------
$ ls -lh
-rw-r--r-- 1 root root  33K Sep  1 18:15
rbd-nbd-0001-0024-fed5480a-f00f-417a-a51d-31d8a8144c03-0000000000000003-d2e89c87-0b4d-11ec-8ea6-160f128e682d.gz

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-09-16 13:55:15 +00:00
Prasanna Kumar Kalever
314516cedd deploy: fix cephLogDir passing to storageclass via helm
cephLogDir: is a storage class option that is passed to rbd-nbd daemon.
cephLogDirHostPath: is a nodeplugin daemonset level option that helps in
                   using the right host-path while bind-mounting

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-09-16 13:55:15 +00:00
Madhu Rajanna
d22e7a1bdb helm: update templates for ceph.conf
updated cephfs and rbd templates to
configure the ceph.conf content.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-09-03 14:14:43 +00:00
Prasanna Kumar Kalever
473adf99fc deploy: provide variable to alter hostpath location for ceph clients
Also update the documentation about the same.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-08-24 07:15:30 +00:00
Prasanna Kumar Kalever
0be7024726 rbd: provide host-path for rbd-nbd logging
Problem:
--------
1. rbd-nbd by default logs to /var/log/ceph/ceph-client.admin.log,
Unfortunately, container doesn't have /var/log/ceph directory hence
rbd-nbd is not logging now.
2. Rbd-nbd logs are not persistent across nodeplugin restarts.

Solution:
--------
Provide a host path so that log directory is made available, and the
logs persist on the hostnode across container restarts.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-08-24 07:15:30 +00:00
Humble Chirammal
3462cd9bbd helm: correct the groupVersion of CSIDriver in the chart
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-08-16 15:21:27 +00:00
Humble Chirammal
8e00c2c810 helm: correct watch verb in topology RBAC
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-08-16 15:21:27 +00:00
Madhu Rajanna
d321663872 deploy: add template changes for mapping
added template changes for the clusterID and
poolID,fsID mapping details for the pod templates.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-08-05 16:07:51 +00:00
Yug Gupta
ea088d40be helm: update rbd provisioner updateStrategy
Update ceph-csi-rbd.provisioner updatestrategy
to allow maxUnavailable pods at a time to be 50%

Signed-off-by: Yug Gupta <yuggupta27@gmail.com>
2021-08-05 14:04:16 +00:00
Alexandre Lossent
5cba04c470 cephfs: support selinux mount options
- mount host's /etc/selinux in node plugins
- process mount options in all code paths for cephfs volume options

Signed-off-by: Alexandre Lossent <alexandre.lossent@cern.ch>
2021-08-04 12:59:34 +00:00
Prasanna Kumar Kalever
b6a88dd728 rbd: add volume healer
Problem:
-------
For rbd nbd userspace mounter backends, after a restart of the nodeplugin
all the mounts will start seeing IO errors. This is because, for rbd-nbd
backends there will be a userspace mount daemon running per volume, post
restart of the nodeplugin pod, there is no way to restore the daemons
back to life.

Solution:
--------
The volume healer is a one-time activity that is triggered at the startup
time of the rbd nodeplugin. It navigates through the list of volume
attachments on the node and acts accordingly.

For now, it is limited to nbd type storage only, but it is flexible and
can be extended in the future for other backend types as needed.

From a few feets above:
This solves a severe problem for nbd backed csi volumes. The healer while
going through the list of volume attachments on the node, if finds the
volume is in attached state and is of type nbd, then it will attempt to
fix the rbd-nbd volumes by sending a NodeStageVolume request with the
required volume attributes like secrets, device name, image attributes,
and etc.. which will finally help start the required rbd-nbd daemons in
the nodeplugin csi-rbdplugin container. This will allow reattaching the
backend images with the right nbd device, thus allowing the applications
to perform IO without any interruptions even after a nodeplugin restart.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-07-16 16:30:58 +00:00
Prasanna Kumar Kalever
10e4eee481 deploy: add few more cluster-roles for rbd nodeplugin
Nodeplugin needs below cluster roles:
persistentvolumes: get
volumeattachments: list, get

These additional permissions are needed by the volume healer. Volume healer
aims at fixing the volume health issues at the very startup time of the
nodeplugin. As part of its operations, volume healer has to run through
the list of volume attachments and understand details about each
persistentvolume.

The later commits will use these additional cluster roles.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-07-16 16:30:58 +00:00
Prasanna Kumar Kalever
874f6629fb rbd: get default plugin path
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-07-16 16:30:58 +00:00
Niels de Vos
8662e01d2c deploy: allow RBD components to get ServiceAccounts
The provisioner and node-plugin have the capability to connect to
Hashicorp Vault with a ServiceAccount from the Namespace where the PVC
is created. This requires permissions to read the contents of the
ServiceAccount from an other Namespace than where Ceph-CSI is deployed.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-07-13 17:16:35 +00:00
Yug
230f0d3f22 helm: Add csi secret template to ceph-csi-rbd
Deploy csi secret when using the helm chart.

Signed-off-by: Yug <yuggupta27@gmail.com>
2021-07-06 10:55:41 +00:00
Yug
ecf768ec41 helm: Add storage class template to ceph-csi-rbd
Deploy storage class when using the helm chart.

Signed-off-by: Yug <yuggupta27@gmail.com>
2021-07-06 10:55:41 +00:00
Rakshith R
d04bfe890f helm: fix k8s version string for csidriver crds semverCompare
Current implementation of semvercompare fails against
pre-release versions. This commit fixes it by using
the entire version string at which csidriver api became GA.

s|">=1.18"|">=1.18.0-beta.1"

Fixes: #2039

Signed-off-by: Rakshith R <rar@redhat.com>
2021-05-25 14:23:33 +00:00
Yati Padia
774e8e4042 util: enable golang profiling
Add support for golang profiling.
Standard tools like go tool pprof and curl
work. example:
$ go tool pprof http://localhost:8080/debug/pprof/profile
$ go tool pprof http://localhost:8080/debug/pprof/heap
$ curl http://localhost:8080/debug/pprof/heap?debug=1

https://golang.org/pkg/net/http/pprof/ contains
more details about the pprof interface.

Fixes: #1699

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-05-25 10:41:22 +00:00
Matthias Neugebauer
3505731c42 Replace deprecated GitVersion with Version
This replaces the deprecated `GitVersion` with `Version`.

See a499b4b179/pkg/chartutil/capabilities.go (L71-L74)

Signed-off-by: Matthias Neugebauer <matthias.neugebauer@uni-muenster.de>
2021-04-26 04:17:13 +00:00
Madhu Rajanna
fba6a2d0c3 deploy: add csidriver object for cephfs and rbd
csidriver object can be created on the kubernetes
for below reason.

If a CSI driver creates a CSIDriver object,
Kubernetes users can easily discover the CSI
Drivers installed on their cluster
(simply by issuing kubectl get CSIDriver)

Ref: https://kubernetes-csi.github.io/docs/csi-driver-object.html#what-is-the-csidriver-object

attachRequired is always required to be set to
true to avoid issue on RWO PVC.

more details about it at https://github.com/rook/rook/pull/4332

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-03-31 13:41:35 +00:00
Niels de Vos
41e4dbe509 deploy: use "devel" branch instead of "master"
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-03-01 10:51:30 +05:30
Madhu Rajanna
7835609b06 set priorityclass on provisioner pods
set system-cluster-critical priorityclass on
provisioner pods. the system-cluster-critical is
having lowest priority compared to node-critical.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-02-19 13:29:09 +00:00
Madhu Rajanna
2190ca922e set priorityclass on plugin pods
set system-node-critical priority on the plugin
pods, as its the highest priority and this need to
be applied on plugin pods as its critical for
storage in cluster.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-02-19 13:29:09 +00:00
Madhu Rajanna
e6098520d1 rbd: add configmap get clusterrole for provisioner
as provisioner need to get the configmap from
different namespace to check tenant configuration.
added the clusterrole get access for the same.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-02-04 14:58:40 +00:00
Matthias Neugebauer
50a720f13d helm: Fix plugin socket path
PR #1736 made the kubelet path configurable. It also introduced a change in
the path to the CSI socket. By default the path is now
`/var/lib/kubelet/cephfs.csi.ceph.com/csi.sock` instead of
`/var/lib/kubelet/plugins/cephfs.csi.ceph.com/csi.sock`. This PR
restores the old default.

Signed-off-by: Matthias Neugebauer <matthias.neugebauer@uni-muenster.de>
2021-02-04 04:57:15 +00:00
Matt Brown
123a26abb3 deploy, helm: enable secret watch in rbac
enables secret ''watch'' rbac permission for ceph-csi-rbd-provisioner role. Fixes 1841.

Signed-off-by: Matt Brown <matthewbrown18@gmail.com>
2021-01-26 15:41:11 +00:00
Niels de Vos
dd29c6c06b deploy: allow rbd nodeplugin to read ConfigMaps from Tenants
Tenants can have their own ConfigMap that contains connection parameters
to the Vault Service where the PV encyption keys are located. It is
possible for a Tenant to use a different Vault Service than the one
configured by the Storage Admin who deployed Ceph-CSI.

For this, the node-plugin needs to be able to read the ConfigMap from
the Tenants namespace.

See-also: docs/design/proposals/encryption-with-vault-tokens.md
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-16 16:45:29 +00:00
Niels de Vos
cb1899b8c0 deploy: allow rbd nodeplugin to read Secrets from Tenants
In order to fetch the Kubernetes Secret with the Vault Token for a
Tenant, the ClusterRole needs to allow reading Secrets from all
Kubernetes Namespaces (each Tenant has their own Namespace).

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-14 14:45:09 +00:00
Niels de Vos
0e6443e4c1 deploy: add --extra-create-metadata arg to csi-provisioner sidecar
This argument in csi-provisioner sidecar allows us to receive pv/pvc
name/namespace metadata in the createVolume() request.

For ex:

    csi.storage.k8s.io/pvc/name
    csi.storage.k8s.io/pvc/namespace
    csi.storage.k8s.io/pv/name

This is a useful information which can be used depend on the use case we
have at our driver. The features like vault token enablement for multi
tenancy, RBD mirroring ..etc can consume this based on the need.

Refer: #1305
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-09 13:58:48 +00:00
Madhu Rajanna
518ccf42b3 deploy: add option to set default-fstype in provisioner
external-provisioner is exposing a new argument
to set the default fstype while starting the provisioner
sidecar, if the fstype is not specified in the storageclass
the default fstype will be applied for the pvc created from
the storageclass.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-08 16:09:25 +00:00
Mudit Agarwal
a92d8d7f2e deploy: update csi sidecar images
This PR makes the changes in csi templates and
upgrade documentation required for updating
csi sidecar images.

Signed-off-by: Mudit Agarwal <muagarwa@redhat.com>
2020-12-08 10:23:34 +00:00
Madhu Rajanna
50d884bad3 helm: make kubelet path configurable
removed unwanted extra arguments from the helm templates
and added a single value kubeletDir to make the kubelet
root-dir configurable.

previously used variables like socketDir,registrationDir
and pluginDir is removed now because if we have the kubelet
path we can derive all other required path for cephcsi to
work properly.

fixes: #1475

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-02 07:16:12 +00:00
Madhu Rajanna
fc9b2e5ac5 deploy: update deployment template for new controller
updated deployment template for the new controller and
also added `update` configmap RBAC for the controller
as the controller uses the configmap for the leader
election.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-11-28 18:50:00 +00:00
Madhu Rajanna
8d3a44d0c4 rbd: add minsnapshotsonimage flag
An rbd image can have a maximum number of
snapshots defined by maxsnapshotsonimage
On the limit is reached the cephcsi will
start flattening the older snapshots and
returns the ABORT error message, The Request
comes after this as to wait till all the
images are flattened (this will increase the
PVC creation time.  Instead of waiting till
the maximum snapshots on an RBD image, we can
have a soft limit, once the limit reached
cephcsi will start flattening the task to
break the chain. With this PVC  creation time
will only be affected when the hard limit
(minsnapshotsonimage) reached.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-11-18 05:59:20 +00:00