Commit Graph

125 Commits

Author SHA1 Message Date
Silvan Loser
059969b10b helm: allowPrivilegeEscalation: true in containerSecurityContext
When running the kubernetes cluster with one single privileged
PodSecurityPolicy which is allowing everything the nodeplugin
daemonset can fail to start. To be precise the problem is the
defaultAllowPrivilegeEscalation: false configuration in the PSP.
 Containers of the nodeplugin daemonset won't start when they
have privileged: true but no allowPrivilegeEscalation in their
container securityContext.

Kubernetes will not schedule if this mismatch exists cannot set
allowPrivilegeEscalation to false and privileged to true

Signed-off-by: Silvan Loser <silvan.loser@hotmail.ch>
Signed-off-by: Silvan Loser <33911078+losil@users.noreply.github.com>
(cherry picked from commit 06c4477ff9)
2022-04-26 10:02:04 +00:00
Madhu Rajanna
0932a4b6be revert: "helm: update image tag for release 3.6.1"
This reverts commit 1bd6297ecb.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-04-22 16:05:51 +00:00
Madhu Rajanna
1bd6297ecb helm: update image tag for release 3.6.1
This commit change the required image tag to
v3.6.1 instead of v3.6-canary for v3.6.1 release

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-04-22 13:41:39 +00:00
Madhu Rajanna
76398d6887 util: Add RBD specific options in clusterInfo
As the netNamespaceFilePath can be separate for
both cephfs and rbd adding the netNamespaceFilePath
path for RBD, This will help us to keep RBD and
CephFS specific options separately.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit 766346868e)
2022-04-19 16:33:59 +00:00
Madhu Rajanna
3161a6b060 util: add support for the nsenter
add support to run rbd map and mount -t
commands with the nsenter.

complete design of pod/multus network
is added here https://github.com/rook/rook/
blob/master/design/ceph/multus-network.md#csi-pods

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit 7b2aef0d81)
2022-04-08 14:44:20 +00:00
Humble Chirammal
2790daac39 deploy: update deployment templates to point to 3.6-canary images
This commit revert the deployment templates in release 3.6 branch
to canary.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-04-06 04:55:57 +00:00
Humble Chirammal
eff3e9a237 helm: update image tag for release 3.6 instead of canary
This commit change the image tag for release v3.6 instead of
canary.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-04-05 14:25:09 +00:00
Rakshith R
40de75e0db rbd: modify oidc token file path according to FHS 3.0
OIDC token file path has been modified from
`/var/run/secrets/token` to `/run/secrets/tokens`.
This has been done to ensure compliance with
FHS 3.0.

refer:
https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch05s13.html

Signed-off-by: Rakshith R <rar@redhat.com>
2022-03-23 13:29:35 +00:00
Rakshith R
4f0bb2315b rbd: add aws-sts-metdata encryption type
With Amazon STS and kubernetes cluster is configured with
OIDC identity provider, credentials to access Amazon KMS
can be fetched using oidc-token(serviceaccount token).
Each tenant/namespace needs to create a secret with aws region,
role and CMK ARN.
Ceph-CSI will assume the given role with oidc token and access
aws KMS, with given CMK to encrypt/decrypt DEK which will stored
in the image metdata.

Refer: https://docs.aws.amazon.com/STS/latest/APIReference/welcome.html
Resolves: #2879

Signed-off-by: Rakshith R <rar@redhat.com>
2022-03-16 07:29:56 +00:00
Madhu Rajanna
dd96024620 doc: remove mention of image feature dependency
avoid specifying the image feature dependencies
and add a link to rbd official document for
reference to the image feature dependencies.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-03-02 09:08:33 +00:00
Madhu Rajanna
d5c98f81a2 rbd: make image features as optional parameter
Makes the rbd images features in the storageclass
as optional so that default image features of librbd
can be used. and also kept the option to user
to specify the image features in the storageclass.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-02-28 13:10:03 +00:00
Madhu Rajanna
fb3835691f rbd: add support for deep-flatten image feature
as deep-flatten is long supported in ceph and its
enabled by default in the librbd, providing an option
to enable it in cephcsi for the rbd images we are
creating.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-02-28 13:10:03 +00:00
Silvio Gissi
9c50e255fb helm: make ceph.conf ConfigMap name configurable
ConfigMap name was hardcoded and led to conflicts. Fixes #2858.

Signed-off-by: Silvio Gissi <silvio@gissilabs.com>
2022-02-21 07:25:22 +00:00
Francesco Astegiano
4235178f7c helm: Add selinuxMount flag to enable/disable /etc/selinux host mount
Add selinuxMount flag to enable/disable /etc/selinux host mount inside pods
to support selinux-enabled filesystems

Signed-off-by: Francesco Astegiano <francesco.astegiano@gmail.com>
2022-02-16 12:48:00 +00:00
Deividas Burškaitis
91c22f521b helm: add port sections to helm templates
to show what ports containers are exposing add port sections to nodeplugin
and provisioner helm templates

Signed-off-by: Deividas Burškaitis <deividas.burskaitis@oxylabs.io>
2022-02-15 10:06:26 +00:00
Madhu Rajanna
0311eb5f44 helm: remove namespace from storageclass yaml
removes namespace from non-namespaced storageclass
object.

fixes: #2714

Replacement for #2715 as we didnt receive any update
and PR is already closed.

Co-authored-by: jhrcz-ls
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-02-11 12:32:58 +00:00
Rakshith R
3203673d17 cleanup: remove ceph.conf WA options which are already fixed
This commit removes ceph.conf WA options:
```
     # Workaround for http://tracker.ceph.com/issues/23446
     fuse_set_user_groups = false

     # ceph-fuse which uses libfuse2 by default has write buffer size of 2KiB
     # adding 'fuse_big_writes = true' option by default to override this limit
     # see https://github.com/ceph/ceph-csi/issues/1928
     fuse_big_writes = true
```
Since they are already fixed.

Refer: https://tracker.ceph.com/issues/44885
Refer: https://tracker.ceph.com/issues/23446
Closes: #2825

Signed-off-by: Rakshith R <rar@redhat.com>
2022-02-04 15:42:32 +00:00
Humble Chirammal
de2489ed7d deploy: update csi-snapshotter sidecar to v5.0.1
This release of snapshotter has a breaking change as mentioned
in the release note:

Refer#
[1]: https://github.com/kubernetes-csi/external-snapshotter/releases/tag/v5.0.0

RBAC rules are also updated with this commit.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-02-03 19:01:57 +00:00
Madhu Rajanna
28fef9b379 cleanup: remove thick provisioning code
This commit removes the thick provisioning
code as thick provisioning is deprecated in
cephcsi 3.5.0.

fixes: #2795

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-01-28 11:17:15 +00:00
Madhu Rajanna
3a445cfc36 deploy: update resizer to 1.4.0
updating external resizer image version
from 1.3.0 to latest available release i.e
1.4.0

1.4.0 changelog link
https://github.com/kubernetes-csi/
external-resizer/blob/master/CHANGELOG/CHANGELOG-1.4.md

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-01-26 15:22:24 +00:00
Humble Chirammal
0078e5c8e7 deploy: update node driver registrar container to v2.4.0
This commit updates the node driver registrar container to latest
version.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-01-14 17:24:49 +05:30
Humble Chirammal
0ab717f06f deploy: update csi-attacher to v3.4.0
This commit update the csi-attacher sidecar version to v3.4.0

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-01-14 17:24:49 +05:30
Humble Chirammal
ea8e360888 deploy: update sidecars to latest versions.
This commit updates sidecars to the latest available version
which is compatible with kubernetes 1.23 and csi spec 1.5

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-01-14 17:24:49 +05:30
Niels de Vos
ee2e97b62d deploy: add CSI-Addons endpoint
Deployments place all sockets for communicating with CSI components in
the shared `/csi` directory. The CSI-Addons socket was introduced
recently, but not configured to be in the same location (by default
placed in `/tmp`).

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-12-22 13:21:59 +00:00
Steven Reitsma
6be0e8cb51 helm: Fix missing ClusterRoleBinding for nodeplugin ServiceAccount
When topology is disabled, the ClusterRoleBinding is not created in the Helm
chart. However, the nodeplugin needs access to volumeattachments for the volume
healer.

Signed-off-by: Steven Reitsma <steven@properchaos.nl>
2021-12-22 11:06:11 +00:00
Konstantin Shalygin
7411773f73 rbd: added RBD features support for krbd
Added support for `object-map, fast-diff`

Signed-off-by: Konstantin Shalygin <k0ste@k0ste.ru>
2021-12-07 07:38:24 +00:00
Toby Jackson
989905aa9f helm: csiplugin-configmap generates invalid configuation
When generating csiconfiguration from values the config.json key gets merged with cluster-mapping.json
as the config.json toYaml element supresses a newline.

This fixes the situation where configuration is generated as shown;

```
 data:
   config.json: |-
    [{"clusterID":"....","monitors":["..."]}]cluster-mapping.json: |-
    []
```

Signed-off-by: Toby Jackson <toby@warmfusion.co.uk>
2021-11-25 06:31:38 +00:00
Rakshith R
c461f9d9b7 helm: use "version: 3-canary" for helm charts on "devel"
Version field for helm Chart.yaml needs to have SemVer 2
compatible value, therefore use "<MAJOR-VERSION>-canary"
on "devel" branch.

Refer: https://helm.sh/docs/topics/charts/#the-chartyaml-file

Signed-off-by: Rakshith R <rar@redhat.com>
2021-11-24 04:32:15 +00:00
Prasanna Kumar Kalever
ec8397d8d9 deploy: updated sc templates for map/unmap Options with the new schema
Fixes: #2641
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-11-23 08:54:37 +00:00
Prasanna Kumar Kalever
9a3170bf77 rbd: provide a way to disable the auto fallback to nbd mounter
This change allows the user to choose not to fallback to NBD mounter
when some ImageFeatures are absent with krbd driver, rather just fail
the NodeStage call.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-11-01 08:17:36 +00:00
Shaohui Liu
af752dd38f helm: support adding annotations to StorageClasses
Signed-off-by: Shaohui Liu <liushaohui@xiaomi.com>
2021-10-28 16:56:12 +00:00
Madhu Rajanna
e45bf03bd8 helm: remove securityContext from cephfs deployment
we dont need securityContext for the cephfs provisioner
pod as its not doing any special operations like mount,
selinux operations etc .

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-09-22 07:12:34 +00:00
Madhu Rajanna
f267d77801 helm: remove extra volumes from rbd plugin PSP
removed extra volume permissions from the rbd
nodeplugin PSP

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-09-22 07:12:34 +00:00
Madhu Rajanna
6f1066fd36 helm: reduce the PSP permission for rbd deployment
rbd deployment doesnot need extra permission like
privileged and extra volumes etc.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-09-22 07:12:34 +00:00
Prasanna Kumar Kalever
c9cc36d8db rbd: provide alternatives to preserve the ceph log files
Currently, we delete the ceph client log file on unmap/detach.

This patch provides additional alternatives for users who would like to
persist the log files.

Strategies:
-----------
`remove`: delete log file on unmap/detach
`compress`: compress the log file to gzip on unmap/detach
`preserve`: preserve the log file in text format

Note that the default strategy will be remove on unmap, and these options
can be tweaked from the storage class

Compression size details example:

On Map: (with debug-rbd=20)
---------
$ ls -lh
-rw-r--r-- 1 root root 526K Sep  1 18:15
rbd-nbd-0001-0024-fed5480a-f00f-417a-a51d-31d8a8144c03-0000000000000003-d2e89c87-0b4d-11ec-8ea6-160f128e682d.log

On unmap:
---------
$ ls -lh
-rw-r--r-- 1 root root  33K Sep  1 18:15
rbd-nbd-0001-0024-fed5480a-f00f-417a-a51d-31d8a8144c03-0000000000000003-d2e89c87-0b4d-11ec-8ea6-160f128e682d.gz

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-09-16 13:55:15 +00:00
Prasanna Kumar Kalever
314516cedd deploy: fix cephLogDir passing to storageclass via helm
cephLogDir: is a storage class option that is passed to rbd-nbd daemon.
cephLogDirHostPath: is a nodeplugin daemonset level option that helps in
                   using the right host-path while bind-mounting

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-09-16 13:55:15 +00:00
Madhu Rajanna
d22e7a1bdb helm: update templates for ceph.conf
updated cephfs and rbd templates to
configure the ceph.conf content.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-09-03 14:14:43 +00:00
Prasanna Kumar Kalever
473adf99fc deploy: provide variable to alter hostpath location for ceph clients
Also update the documentation about the same.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-08-24 07:15:30 +00:00
Prasanna Kumar Kalever
0be7024726 rbd: provide host-path for rbd-nbd logging
Problem:
--------
1. rbd-nbd by default logs to /var/log/ceph/ceph-client.admin.log,
Unfortunately, container doesn't have /var/log/ceph directory hence
rbd-nbd is not logging now.
2. Rbd-nbd logs are not persistent across nodeplugin restarts.

Solution:
--------
Provide a host path so that log directory is made available, and the
logs persist on the hostnode across container restarts.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-08-24 07:15:30 +00:00
Humble Chirammal
763387c8e2 rebase: update external-resizer to v1.3.0 release
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-08-18 17:05:22 +00:00
Humble Chirammal
68bbd58045 rebase: update sidecars to latest versions
external-provisioner: v2.3.0
external-attacher: v3.3.0
external-snapshotter: v4.2.0
node-driver-registrar: v2.3.0

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-08-17 10:58:52 +00:00
Humble Chirammal
3462cd9bbd helm: correct the groupVersion of CSIDriver in the chart
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-08-16 15:21:27 +00:00
Humble Chirammal
8e00c2c810 helm: correct watch verb in topology RBAC
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-08-16 15:21:27 +00:00
Madhu Rajanna
d321663872 deploy: add template changes for mapping
added template changes for the clusterID and
poolID,fsID mapping details for the pod templates.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-08-05 16:07:51 +00:00
Yug Gupta
ea088d40be helm: update rbd provisioner updateStrategy
Update ceph-csi-rbd.provisioner updatestrategy
to allow maxUnavailable pods at a time to be 50%

Signed-off-by: Yug Gupta <yuggupta27@gmail.com>
2021-08-05 14:04:16 +00:00
Alexandre Lossent
5cba04c470 cephfs: support selinux mount options
- mount host's /etc/selinux in node plugins
- process mount options in all code paths for cephfs volume options

Signed-off-by: Alexandre Lossent <alexandre.lossent@cern.ch>
2021-08-04 12:59:34 +00:00
Prasanna Kumar Kalever
b6a88dd728 rbd: add volume healer
Problem:
-------
For rbd nbd userspace mounter backends, after a restart of the nodeplugin
all the mounts will start seeing IO errors. This is because, for rbd-nbd
backends there will be a userspace mount daemon running per volume, post
restart of the nodeplugin pod, there is no way to restore the daemons
back to life.

Solution:
--------
The volume healer is a one-time activity that is triggered at the startup
time of the rbd nodeplugin. It navigates through the list of volume
attachments on the node and acts accordingly.

For now, it is limited to nbd type storage only, but it is flexible and
can be extended in the future for other backend types as needed.

From a few feets above:
This solves a severe problem for nbd backed csi volumes. The healer while
going through the list of volume attachments on the node, if finds the
volume is in attached state and is of type nbd, then it will attempt to
fix the rbd-nbd volumes by sending a NodeStageVolume request with the
required volume attributes like secrets, device name, image attributes,
and etc.. which will finally help start the required rbd-nbd daemons in
the nodeplugin csi-rbdplugin container. This will allow reattaching the
backend images with the right nbd device, thus allowing the applications
to perform IO without any interruptions even after a nodeplugin restart.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-07-16 16:30:58 +00:00
Prasanna Kumar Kalever
10e4eee481 deploy: add few more cluster-roles for rbd nodeplugin
Nodeplugin needs below cluster roles:
persistentvolumes: get
volumeattachments: list, get

These additional permissions are needed by the volume healer. Volume healer
aims at fixing the volume health issues at the very startup time of the
nodeplugin. As part of its operations, volume healer has to run through
the list of volume attachments and understand details about each
persistentvolume.

The later commits will use these additional cluster roles.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-07-16 16:30:58 +00:00
Prasanna Kumar Kalever
874f6629fb rbd: get default plugin path
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-07-16 16:30:58 +00:00
Niels de Vos
8662e01d2c deploy: allow RBD components to get ServiceAccounts
The provisioner and node-plugin have the capability to connect to
Hashicorp Vault with a ServiceAccount from the Namespace where the PVC
is created. This requires permissions to read the contents of the
ServiceAccount from an other Namespace than where Ceph-CSI is deployed.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-07-13 17:16:35 +00:00