Commit Graph

340 Commits

Author SHA1 Message Date
Rakshith R
54fe4158a0 deploy: update fsGroupPolicy to File for nfs driver object
NFS volume similar to CephFs volumes can support
fsGroupPolicy as File, now Kubernetes Kubernetes may
use fsGroup to change permissions and ownership of the
volume to match user requested fsGroup in the pod's
SecurityPolicy regardless of fstype or access mode.

Signed-off-by: Rakshith R <rar@redhat.com>
2022-08-09 13:36:03 +00:00
Rakshith R
48d66d6cfd deploy: modify nfs daemonset to use cephcsi nfs nodeserver
This commit makes modification to nfs daemonset to use
nfs nodeserver. `nfs.NetNamespaceFilePath` example is
added.

Signed-off-by: Rakshith R <rar@redhat.com>
2022-08-09 13:36:03 +00:00
Yati Padia
f0074a3ebf deploy: enable HonorPVReclaimPolicy feature gate
This commit enables the HonorPVReclaimPolicy feature
gate.

fixes: #3230

Signed-off-by: Yati Padia <ypadia@redhat.com>
2022-08-03 19:43:16 +00:00
Prasanna Kumar Kalever
c0a566b5ed deploy: add setmetadata=true in the templates
setmetadata on the volume by default, otherwise e2e will fail

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2022-08-01 07:15:29 +00:00
Prasanna Kumar Kalever
f84265fdf5 deploy: add --extra-create-metadata arg to csi-snapshotter sidecar
This argument in csi-snapshotter sidecar allows us to receive
snapshot-name/snapshot-namespace/snapshotcontent-name metadata in the
CreateSnapshot() request.

For ex:

csi.storage.k8s.io/volumesnapshot/name
csi.storage.k8s.io/volumesnapshot/namespace
csi.storage.k8s.io/volumesnapshotcontent/name

This is a useful information which can be used depend on the use case we
have at our driver. The features like adding metadata to snapshot image
can consume this based on the need.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2022-07-28 19:37:23 +00:00
Humble Chirammal
76ddf8e306 deploy: introduce new log level for sidecar controllers
At present we have single log level configuration for all the containers
running for our CSI pods, which has been defaulted to log Level 5.
However this cause many logs to be spitted in a cluster and cause log
spamming to an extent. This commit introduce one more log level control
for CSI pods called sidecarLogLevel which defaults to log Level 1.

The sidecar controllers like snapshotter, resizer, attacher..etc has
been configured with this new log level and driver pods are with old
configruation value.

This allow us to have different configuration options for sidecar
constrollers and driver pods.

With this, we will also have a choice of different configuation setting
instead of locking onto one variable for the containers deployed via CSI driver.

To summarize the CSI containers maintained by Ceph CSI driver has log
level 5 and controllers/sidecars not maintained by Ceph CSI driver has
log level 1 configuration.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-07-28 08:31:37 +00:00
Rakshith R
5ed305850f build: resolve a fixme and disable tcmu repo
Cmd to disable apache arrow repo is removed, since
it is no longer needed.
Cmd to disable tcmu repo is added to make build pass.

refer: https://github.com/ceph/ceph-container/issues/2034

Signed-off-by: Rakshith R <rar@redhat.com>
2022-07-20 09:29:35 +00:00
Yati Padia
b0b0e083ad cephfs: add update rbac rule to pv resource
This commit adds the update rbac rule to persistent
volume resource as the ci was failing with below error:
cannot update resource "persistentvolumes" in API group
"" at the cluster scope

Signed-off-by: Yati Padia <ypadia@redhat.com>
2022-07-19 14:42:21 +00:00
Yati Padia
776821f17f deploy: update csi-provisioner to latest version
This commits updates csi-provisioner sidecar to
latest version i.e., v3.2.0.

fixes: #3184

Signed-off-by: Yati Padia <ypadia@redhat.com>
2022-07-19 14:42:21 +00:00
Humble Chirammal
1856647506 cephfs: go with default permissions while creating subvolumes
While creating subvolumes, CephFS driver set the mode to `777`
and pass it along to go ceph apis which cause the subvolume
permission to be on 777, however if we create a subvolume
directly in the ceph cluster, the default permission bits are
set which is 755 for the subvolume. This commit try to stick
to the default behaviour even while creating the subvolume.

This also means that we can work with fsgrouppolicy set to
`File` in csiDriver object which is also addressed in this commit.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-07-13 06:49:58 +00:00
Humble Chirammal
08b42e5d67 nfs: make use of latest sidecars in the deployment
The sidecars in the NFS deployment has latest versions which is
also updated for RBD and CephFS drivers. This commit update
the versions in the NFS deployment too.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-07-05 18:06:37 +00:00
Prasanna Kumar Kalever
dc738b96b4 deploy: add setmetadata=true in the templates
setmetadata on the volume by default, otherwise e2e will fail

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2022-06-28 19:12:53 +00:00
Prasanna Kumar Kalever
d3650ae863 deploy: fix the staging path accordingly in the templates
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2022-06-24 12:23:29 +00:00
Humble Chirammal
fa0da71ce2 deploy: update external resizer to v1.5.0
Refer#
https://github.com/kubernetes-csi/external-resizer/releases/tag/v1.5.0

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-06-21 07:43:15 +00:00
Humble Chirammal
34378aafb8 deploy: make use of latest attacher release v3.5.0
attacher sidecar has a new release:
https://github.com/kubernetes-csi/external-attacher/releases/tag/v3.5.0

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-06-17 15:46:09 +00:00
Madhu Rajanna
7a2dd4c3cf rbd: create token and use it for vault SA
create the token if kubernetes version in
1.24+ and use it for vault sa.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
Signed-off-by: Rakshith R <rar@redhat.com>
2022-06-17 11:37:59 +00:00
Humble Chirammal
8d3bb82949 deploy: remove attachrequired param from csidriver object
As the attacher is no longer required we have to mention the same
for csidriver object parameter.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-06-06 12:25:11 +00:00
Humble Chirammal
c0fa88435f deploy: remove external-attacher sidecar from cephfs deployment
CephFS CSI driver dont need attacher sidecar for its operations.
This commit remove the same. The RBAC has also got adjusted.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-06-06 12:25:11 +00:00
Humble Chirammal
260f4b6567 deploy: update external-snapshotter to v6.0.1
The extenal snapshotter has a new version which address some of the
issues in v6.0.0.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-05-31 15:37:14 +00:00
Madhu Rajanna
3c1ae4c5c8 deploy: update snapshotter version to v6
external-snapshotter v6.0.0 is released updating
the snapshotter to v6.0.0

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-05-31 07:23:49 +00:00
Humble Chirammal
4e02dc1da9 deploy: Enable featuregate for volume expansion recovery
The enabled feature gates helps us to recover from volume expansion
failure.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-05-25 18:03:16 +00:00
Humble Chirammal
5e26043d61 deploy: update node-driver-registrar sidecar to latest
updated deployment yamls for the latest version.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-05-25 07:27:03 +00:00
Rakshith R
5983d5730a deploy: add support for nfs snapshot
This commit add external-snapshotter container and
required rbac to support nfs snaphots.
Example volumesnaphotclass, volumesnapshot,
pvc-restore and pod-restore yamls are also added.

Signed-off-by: Rakshith R <rar@redhat.com>
2022-05-24 18:13:02 +00:00
Humble Chirammal
ba93a2b037 cephfs: adjust RBAC for CephFS provisioner role
CephFS CSI driver does not need permissions on Node,ConfigMap objects.
This PR also adjust the update->Patch for a couple of RBAC.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-05-23 21:52:33 +00:00
Niels de Vos
891a840aa6 deploy: add resize support to NFS-provisioner deployment
Resizing is handled by the csi-resizer container, which needs to run in
the provisioner Pod. In addition to the container, the StorageClass also
needs to allow volume expansion.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-05-10 17:43:59 +00:00
Niels de Vos
1818529a4c deploy: allow hostPort 29653 for NFS node-plugin
This should address the following failure when Pod Security Policies are
enabled:

> FailedCreate: Error creating: pods "csi-nfs-node-" is forbidden:
> PodSecurityPolicy: unable to admit pod: spec.containers[2].hostPort:
> Invalid value: 29653: Host port 29653 is not allowed to be used.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-05-10 00:43:43 +00:00
Niels de Vos
fc1529f268 rebase: bump nfsplugin to v4.0.0
Several bugs have been fixed and new features have been added.

See-also: https://github.com/kubernetes-csi/csi-driver-nfs/releases/tag/v4.0.0
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-05-09 06:34:49 +00:00
Rakshith R
c880061882 ci: use canary csi-provisioner image to test different sc clones
This commit is added to use canary csi-provisioner image
to test different sc pvc-pvc cloning feature, which is not
yet present in released versions.
refer:
https://github.com/kubernetes-csi/external-provisioner/pull/699

Signed-off-by: Rakshith R <rar@redhat.com>
2022-05-06 10:32:21 +00:00
Humble Chirammal
188e560ee9 nfs: use latest liveness probe and node driver registrar
This commit make use of latest sidecars of livenessprobe and
node driver registrar in NFS driver deployment.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-05-04 17:38:54 +00:00
Humble Chirammal
b50e93e689 nfs: remove node plugin RBAC for NFS provisioner
this commit removes the node plugin RBAC for NFS plugin as it is
not needed.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-04-27 15:40:48 +00:00
Humble Chirammal
a2059d5cb2 cephfs: remove nodeplugin RBAC
This commit remove the clusterRole and Binding of cephfs node plugin
as the node RBAC is not needed for CephFS.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-04-27 10:51:33 +00:00
Silvan Loser
f2e0fa28fb deploy: allowPrivilegeEscalation: true in containerSecurityContext
When running the kubernetes cluster with one single privileged
PodSecurityPolicy which is allowing everything the nodeplugin
daemonset can fail to start. To be precise the problem is the
defaultAllowPrivilegeEscalation: false configuration in the PSP.
 Containers of the nodeplugin daemonset won't start when they
have privileged: true but no allowPrivilegeEscalation in their
container securityContext.

Kubernetes will not schedule if this mismatch exists cannot set
allowPrivilegeEscalation to false and privileged to true:

Signed-off-by: Silvan Loser <silvan.loser@hotmail.ch>
Signed-off-by: Silvan Loser <33911078+losil@users.noreply.github.com>
2022-04-22 23:36:02 +00:00
Humble Chirammal
7d3fd4f683 nfs: change the image registry for sidecars
This commit change the image registry URL for sidecars in the
NFS deployment from `k8s.gcr.io` to `registry.k8s.io` as
the migration is happening from former to the latter.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-04-20 10:05:13 +00:00
Humble Chirammal
6d06698672 rbd: change the image registry for sidecars
This commit change the image registry URL for sidecars in the
RBD deployment from `k8s.gcr.io` to `registry.k8s.io` as
the migration is happening from former to the latter.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-04-20 10:05:13 +00:00
Humble Chirammal
1ced736447 cephfs: change the image registry for sidecars
This commit change the image registry URL for sidecars in the
CephFS deployment from `k8s.gcr.io` to `registry.k8s.io` as
the migration is happening from former to the latter.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-04-20 10:05:13 +00:00
Niels de Vos
6b34e6c899 deploy: use k8s.gcr.io registry for the NFS-nodeplugin
Kubernetes CSI now hosts the container-image for the NFS-nodeplugin in
the the k8s.gcr.io instead of the Microsoft registry.

See-also: kubernetes-csi/csi-driver-nfs@7b5b6f344
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-04-14 09:06:49 +00:00
Rakshith R
784b086ea5 nfs: add provisioner & plugin sa to scc.yaml
This commit adds nfs provisioner & plugin sa to
scc.yaml to be used with openshift.

Signed-off-by: Rakshith R <rar@redhat.com>
2022-04-13 09:14:15 +00:00
Prasanna Kumar Kalever
d870cb152a deploy: add --extra-create-metadata arg to csi-snapshotter sidecar
This argument in csi-snapshotter sidecar allows us to receive
snapshot-name/snapshot-namespace/snapshotcontent-name metadata in the
CreateSnapshot() request.

For ex:

csi.storage.k8s.io/volumesnapshot/name
csi.storage.k8s.io/volumesnapshot/namespace
csi.storage.k8s.io/volumesnapshotcontent/name

This is a useful information which can be used depend on the use case we
have at our driver. The features like adding metadata to snapshot image
can consume this based on the need.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2022-04-08 15:43:14 +00:00
Madhu Rajanna
7b2aef0d81 util: add support for the nsenter
add support to run rbd map and mount -t
commands with the nsenter.

complete design of pod/multus network
is added here https://github.com/rook/rook/
blob/master/design/ceph/multus-network.md#csi-pods

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-04-08 10:23:21 +00:00
Niels de Vos
804e2715d8 deploy: add deployment artifacts for NFS support
These deployment files are heavily based on the CephFS deployment.

Deploying an environment with these files work for me in minikube. This
should make it possible to add e2e testing as well.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-04-01 10:37:41 +00:00
Niels de Vos
87f87141be deploy: add CSIDriver for NFS
The API is extended for generation of the NFS CSIDriver object. The
YAML file under deploy/ was created by `yamlgen`.

The contents of the csidriver.yaml file is heavily based on the upstream
CSIDriver from the Kubernetes csi-driver-nfs project.

Because ./tools/yamlgen uses the API, it gets copied under vendor/ .
This causes two copies of the API to be included in the repository, but
that can not be prevented, it seems.

See-also: https://github.com/kubernetes-csi/csi-driver-nfs/blob/master/deploy/csi-nfs-driverinfo.yaml
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-03-28 11:23:17 +00:00
Humble Chirammal
16abbbc846 build: remove cache while building container image
Reduce size of the container image by removing the cache in deploy
and devel container.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-03-28 06:09:27 +00:00
Rakshith R
40de75e0db rbd: modify oidc token file path according to FHS 3.0
OIDC token file path has been modified from
`/var/run/secrets/token` to `/run/secrets/tokens`.
This has been done to ensure compliance with
FHS 3.0.

refer:
https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch05s13.html

Signed-off-by: Rakshith R <rar@redhat.com>
2022-03-23 13:29:35 +00:00
Rakshith R
4f0bb2315b rbd: add aws-sts-metdata encryption type
With Amazon STS and kubernetes cluster is configured with
OIDC identity provider, credentials to access Amazon KMS
can be fetched using oidc-token(serviceaccount token).
Each tenant/namespace needs to create a secret with aws region,
role and CMK ARN.
Ceph-CSI will assume the given role with oidc token and access
aws KMS, with given CMK to encrypt/decrypt DEK which will stored
in the image metdata.

Refer: https://docs.aws.amazon.com/STS/latest/APIReference/welcome.html
Resolves: #2879

Signed-off-by: Rakshith R <rar@redhat.com>
2022-03-16 07:29:56 +00:00
Robert Vasek
80dda7cc30 cephfs: detect corrupt ceph-fuse mounts and try to remount
Mounts managed by ceph-fuse may get corrupted by e.g. the ceph-fuse process
exiting abruptly, or its parent container being terminated, taking down its
child processes with it.

This commit adds checks to NodeStageVolume and NodePublishVolume procedures
to detect whether a mountpoint in staging_target_path and/or target_path is
corrupted, and remount is performed if corruption is detected.

Signed-off-by: Robert Vasek <robert.vasek@cern.ch>
2022-03-10 06:05:52 +00:00
Niels de Vos
cbec296543 build: disable removed Apache Arrow repository
The CentOS 8 repository for Apache Arrow has been removed. This causes
container-image builds fail with the following error:

    Errors during downloading metadata for repository 'apache-arrow-centos':
      - Status code: 404 for https://apache.jfrog.io/artifactory/arrow/centos/8/x86_64/repodata/repomd.xml (IP: 54.190.66.70)
    Error: Failed to download metadata for repo 'apache-arrow-centos': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried

The Ceph base image has `arrow/centos/8` configured, maybe Apache Arrow
offers a CentOS Stream 8 repository now? Once the Ceph container-image
has been updated, the repository can be enabled again.

Ceph-CSI does not depend on Apache Arrow, so there is no functional
change by disabling the repository.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-02-04 10:23:58 +00:00
Humble Chirammal
de2489ed7d deploy: update csi-snapshotter sidecar to v5.0.1
This release of snapshotter has a breaking change as mentioned
in the release note:

Refer#
[1]: https://github.com/kubernetes-csi/external-snapshotter/releases/tag/v5.0.0

RBAC rules are also updated with this commit.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-02-03 19:01:57 +00:00
Madhu Rajanna
3a445cfc36 deploy: update resizer to 1.4.0
updating external resizer image version
from 1.3.0 to latest available release i.e
1.4.0

1.4.0 changelog link
https://github.com/kubernetes-csi/
external-resizer/blob/master/CHANGELOG/CHANGELOG-1.4.md

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-01-26 15:22:24 +00:00
Humble Chirammal
0078e5c8e7 deploy: update node driver registrar container to v2.4.0
This commit updates the node driver registrar container to latest
version.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-01-14 17:24:49 +05:30
Humble Chirammal
0ab717f06f deploy: update csi-attacher to v3.4.0
This commit update the csi-attacher sidecar version to v3.4.0

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-01-14 17:24:49 +05:30
Humble Chirammal
ea8e360888 deploy: update sidecars to latest versions.
This commit updates sidecars to the latest available version
which is compatible with kubernetes 1.23 and csi spec 1.5

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-01-14 17:24:49 +05:30
Niels de Vos
ee2e97b62d deploy: add CSI-Addons endpoint
Deployments place all sockets for communicating with CSI components in
the shared `/csi` directory. The CSI-Addons socket was introduced
recently, but not configured to be in the same location (by default
placed in `/tmp`).

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-12-22 13:21:59 +00:00
Niels de Vos
cff0e04e3c build: remove unneeded empty YAML document from deployment artifacts
The generated files under the deploy/ directory contain an empty YAML
document that may cause confusion for some versions of kubectl. Dropping
the unneeded `---` start of the file for the header should make parsing
of the deployment artifacts a little less error prone.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-10-15 16:08:59 +00:00
Niels de Vos
c443320126 deploy: move rbd/ceph-csi-config ConfigMap to API
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-10-15 16:08:59 +00:00
Niels de Vos
584d43a132 deploy: move rbd/CSIDriver to API
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-10-15 16:08:59 +00:00
Niels de Vos
5ea99fdd5b build: add yamlgen to build deployment files
This initial version of yamlgen generates deploy/scc.yaml based on the
deployment artifact that is provided by the new api/deploy/ocp package.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-10-05 11:26:50 +00:00
Madhu Rajanna
9bff7b0ac9 deploy: remove extra volumes from cephfs plugin PSP
removed extra volume permissions from the cephfs
nodeplugin PSP.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-09-22 07:12:34 +00:00
Madhu Rajanna
051af3b257 deploy: reduce the PSP permission for cephfs deployment
cephfs deployment doesnot need extra permission like
privileged,Capabilities and remove unwanted volumes.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-09-22 07:12:34 +00:00
Madhu Rajanna
7fc1bf1321 deploy: remove extra volumes from rbd plugin PSP
removed extra volume permissions from the rbd
nodeplugin PSP.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-09-22 07:12:34 +00:00
Madhu Rajanna
9e88fd1eb7 deploy: reduce the PSP permission for rbd deployment
rbd deployment doesnot need extra permission like
privileged,Capabilities and remove unwanted volumes.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-09-22 07:12:34 +00:00
Madhu Rajanna
e5569f0547 deploy: remove securityContext from rbd provisioner
we dont need securityContext for the rbd provisioner
pod as its not doing any special operations like map
,unmap selinux etc.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-09-22 07:12:34 +00:00
Madhu Rajanna
f1c64a2a6b deploy: remove securityContext from cephfs provisioner
we dont need securityContext for the cephfs provisioner
pod as its not doing any special operations like mounts,
selinux etc.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-09-22 07:12:34 +00:00
Madhu Rajanna
cc6c51395e deploy: update templates for ceph.conf
updated cephfs and rbd templates to mount
the ceph.conf configmap.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-09-03 14:14:43 +00:00
Prasanna Kumar Kalever
0be7024726 rbd: provide host-path for rbd-nbd logging
Problem:
--------
1. rbd-nbd by default logs to /var/log/ceph/ceph-client.admin.log,
Unfortunately, container doesn't have /var/log/ceph directory hence
rbd-nbd is not logging now.
2. Rbd-nbd logs are not persistent across nodeplugin restarts.

Solution:
--------
Provide a host path so that log directory is made available, and the
logs persist on the hostnode across container restarts.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-08-24 07:15:30 +00:00
Humble Chirammal
763387c8e2 rebase: update external-resizer to v1.3.0 release
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-08-18 17:05:22 +00:00
Humble Chirammal
68bbd58045 rebase: update sidecars to latest versions
external-provisioner: v2.3.0
external-attacher: v3.3.0
external-snapshotter: v4.2.0
node-driver-registrar: v2.3.0

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-08-17 10:58:52 +00:00
Alexandre Lossent
5cba04c470 cephfs: support selinux mount options
- mount host's /etc/selinux in node plugins
- process mount options in all code paths for cephfs volume options

Signed-off-by: Alexandre Lossent <alexandre.lossent@cern.ch>
2021-08-04 12:59:34 +00:00
rtsp
af1f50ba04 deploy: rbd kubernetes manifests
add ability to deploy ceph-csi-rbd on non-default namespace

Signed-off-by: rtsp <git@rtsp.us>
2021-07-31 03:09:14 +00:00
Thomas Kooi
75b9b9fe6d cleanup: fix beta apiVersion for csidriver
This change resolves a typo for installing the CSIDriver
resource in Kubernetes clusters before 1.18,
where the apiVersion is incorrect.

See also:
https://kubernetes-csi.github.io/docs/csi-driver-object.html

[ndevos: replace v1betav1 in examples with v1beta1]
Signed-off-by: Thomas Kooi <t.j.kooi@avisi.nl>
2021-07-22 09:12:44 +00:00
Prasanna Kumar Kalever
b6a88dd728 rbd: add volume healer
Problem:
-------
For rbd nbd userspace mounter backends, after a restart of the nodeplugin
all the mounts will start seeing IO errors. This is because, for rbd-nbd
backends there will be a userspace mount daemon running per volume, post
restart of the nodeplugin pod, there is no way to restore the daemons
back to life.

Solution:
--------
The volume healer is a one-time activity that is triggered at the startup
time of the rbd nodeplugin. It navigates through the list of volume
attachments on the node and acts accordingly.

For now, it is limited to nbd type storage only, but it is flexible and
can be extended in the future for other backend types as needed.

From a few feets above:
This solves a severe problem for nbd backed csi volumes. The healer while
going through the list of volume attachments on the node, if finds the
volume is in attached state and is of type nbd, then it will attempt to
fix the rbd-nbd volumes by sending a NodeStageVolume request with the
required volume attributes like secrets, device name, image attributes,
and etc.. which will finally help start the required rbd-nbd daemons in
the nodeplugin csi-rbdplugin container. This will allow reattaching the
backend images with the right nbd device, thus allowing the applications
to perform IO without any interruptions even after a nodeplugin restart.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-07-16 16:30:58 +00:00
Prasanna Kumar Kalever
10e4eee481 deploy: add few more cluster-roles for rbd nodeplugin
Nodeplugin needs below cluster roles:
persistentvolumes: get
volumeattachments: list, get

These additional permissions are needed by the volume healer. Volume healer
aims at fixing the volume health issues at the very startup time of the
nodeplugin. As part of its operations, volume healer has to run through
the list of volume attachments and understand details about each
persistentvolume.

The later commits will use these additional cluster roles.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-07-16 16:30:58 +00:00
Prasanna Kumar Kalever
874f6629fb rbd: get default plugin path
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-07-16 16:30:58 +00:00
Niels de Vos
8662e01d2c deploy: allow RBD components to get ServiceAccounts
The provisioner and node-plugin have the capability to connect to
Hashicorp Vault with a ServiceAccount from the Namespace where the PVC
is created. This requires permissions to read the contents of the
ServiceAccount from an other Namespace than where Ceph-CSI is deployed.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-07-13 17:16:35 +00:00
Yug
a852e66133 deploy: update node-driver-registrar
update node-driver-registrar to
the latest available release version.

Signed-off-by: Yug <yuggupta27@gmail.com>
2021-06-25 15:45:14 +00:00
Yug
b19717ef99 deploy: update csi-snapshotter sidecar
update csi-snapshotter sidecar to
the latest available release version.

Signed-off-by: Yug <yuggupta27@gmail.com>
2021-06-25 15:45:14 +00:00
Yug
77f2db8875 deploy: update csi-resizer sidecar
update csi-resizer sidecar to
the latest available release version.

Signed-off-by: Yug <yuggupta27@gmail.com>
2021-06-25 15:45:14 +00:00
Yug
fac20a9446 deploy: update csi-attacher sidecar
update csi-attacher sidecar to
the latest available release version.

Signed-off-by: Yug <yuggupta27@gmail.com>
2021-06-25 15:45:14 +00:00
Yug
cf63be41c8 deploy: update csi-provisioner sidecar
update csi-provisioner sidecar to
the latest available release version.

Signed-off-by: Yug <yuggupta27@gmail.com>
2021-06-25 15:45:14 +00:00
Yati Padia
774e8e4042 util: enable golang profiling
Add support for golang profiling.
Standard tools like go tool pprof and curl
work. example:
$ go tool pprof http://localhost:8080/debug/pprof/profile
$ go tool pprof http://localhost:8080/debug/pprof/heap
$ curl http://localhost:8080/debug/pprof/heap?debug=1

https://golang.org/pkg/net/http/pprof/ contains
more details about the pprof interface.

Fixes: #1699

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-05-25 10:41:22 +00:00
Humble Chirammal
1b0ebc43d4 deploy: use serviceAccountName instead of serviceAccount in yamls
serviceAccount is the depricated alias for serviceAccountName, so it
is recommended/suggested to use serviceAccountName instead.

For ex. reference:
https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-04-06 09:00:35 +00:00
Madhu Rajanna
fba6a2d0c3 deploy: add csidriver object for cephfs and rbd
csidriver object can be created on the kubernetes
for below reason.

If a CSI driver creates a CSIDriver object,
Kubernetes users can easily discover the CSI
Drivers installed on their cluster
(simply by issuing kubectl get CSIDriver)

Ref: https://kubernetes-csi.github.io/docs/csi-driver-object.html#what-is-the-csidriver-object

attachRequired is always required to be set to
true to avoid issue on RWO PVC.

more details about it at https://github.com/rook/rook/pull/4332

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-03-31 13:41:35 +00:00
Prasanna Kumar Kalever
ea4489da68 deploy: bump the snapshotter sidecar image version to v4.0.0
use the latest version of csi-snapshotter sidecar image at the
provisioner templates

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-03-15 15:11:01 +00:00
Niels de Vos
06d5d8f23a build: libcephfs-devel is not needed
go-ceph does not  use CephFS development headers, so there is no need to
install libcephfs-devel.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-03-10 09:26:00 +00:00
Madhu Rajanna
7835609b06 set priorityclass on provisioner pods
set system-cluster-critical priorityclass on
provisioner pods. the system-cluster-critical is
having lowest priority compared to node-critical.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-02-19 13:29:09 +00:00
Madhu Rajanna
2190ca922e set priorityclass on plugin pods
set system-node-critical priority on the plugin
pods, as its the highest priority and this need to
be applied on plugin pods as its critical for
storage in cluster.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-02-19 13:29:09 +00:00
Madhu Rajanna
e6098520d1 rbd: add configmap get clusterrole for provisioner
as provisioner need to get the configmap from
different namespace to check tenant configuration.
added the clusterrole get access for the same.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-02-04 14:58:40 +00:00
Matt Brown
123a26abb3 deploy, helm: enable secret watch in rbac
enables secret ''watch'' rbac permission for ceph-csi-rbd-provisioner role. Fixes 1841.

Signed-off-by: Matt Brown <matthewbrown18@gmail.com>
2021-01-26 15:41:11 +00:00
Niels de Vos
dd29c6c06b deploy: allow rbd nodeplugin to read ConfigMaps from Tenants
Tenants can have their own ConfigMap that contains connection parameters
to the Vault Service where the PV encyption keys are located. It is
possible for a Tenant to use a different Vault Service than the one
configured by the Storage Admin who deployed Ceph-CSI.

For this, the node-plugin needs to be able to read the ConfigMap from
the Tenants namespace.

See-also: docs/design/proposals/encryption-with-vault-tokens.md
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-16 16:45:29 +00:00
Madhu Rajanna
b3fbcb9c95 rbd: read configuration from the configmap
if the kms encryption configmap is not mounted
as a volume to the CSI pods, add the code to
read the configuration from the kubernetes. Later
the code to fetch the configmap will be moved to
the new sidecar which is will talk to respective
CO to fetch the encryption configurations.

The k8s configmap uses the standard vault spefic
names to add the configurations. this will be converted
back to the CSI configurations.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-16 15:05:05 +00:00
Niels de Vos
cb1899b8c0 deploy: allow rbd nodeplugin to read Secrets from Tenants
In order to fetch the Kubernetes Secret with the Vault Token for a
Tenant, the ClusterRole needs to allow reading Secrets from all
Kubernetes Namespaces (each Tenant has their own Namespace).

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-14 14:45:09 +00:00
Niels de Vos
0e6443e4c1 deploy: add --extra-create-metadata arg to csi-provisioner sidecar
This argument in csi-provisioner sidecar allows us to receive pv/pvc
name/namespace metadata in the createVolume() request.

For ex:

    csi.storage.k8s.io/pvc/name
    csi.storage.k8s.io/pvc/namespace
    csi.storage.k8s.io/pv/name

This is a useful information which can be used depend on the use case we
have at our driver. The features like vault token enablement for multi
tenancy, RBD mirroring ..etc can consume this based on the need.

Refer: #1305
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-09 13:58:48 +00:00
Madhu Rajanna
518ccf42b3 deploy: add option to set default-fstype in provisioner
external-provisioner is exposing a new argument
to set the default fstype while starting the provisioner
sidecar, if the fstype is not specified in the storageclass
the default fstype will be applied for the pvc created from
the storageclass.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-08 16:09:25 +00:00
Madhu Rajanna
e3d1ba7703 deploy: set topology=false in provisioner sidecar
with csi-provisioner v2.x the topology based
provisioning will not have any backward compatibility
with older version of kubernetes, if the nodes are
not labeled with topology keys, the pvc creation
is going to get fail with error `accessibility
requirements: no available topology found`, disabling
the topology based provisioning by default, if user want
to use it he can always enable it.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-08 16:09:25 +00:00
Mudit Agarwal
a92d8d7f2e deploy: update csi sidecar images
This PR makes the changes in csi templates and
upgrade documentation required for updating
csi sidecar images.

Signed-off-by: Mudit Agarwal <muagarwa@redhat.com>
2020-12-08 10:23:34 +00:00
Madhu Rajanna
39b1f2b4d3 cleanup: fix mispell words
fixed mispell words in the repo.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-11-29 12:47:46 +05:30
Madhu Rajanna
fc9b2e5ac5 deploy: update deployment template for new controller
updated deployment template for the new controller and
also added `update` configmap RBAC for the controller
as the controller uses the configmap for the leader
election.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-11-28 18:50:00 +00:00
Niels de Vos
b26d33b7c1 build: install git as when building from Dockerfile
When running a simple build with only the required arguments, the
following warning are reported:

    $ buildah bud --build-arg=BASE_IMAGE=ceph/ceph:v15 --build-arg=GO_ARCH=amd64 -f ./deploy/cephcsi/image/Dockerfile .
    ...
    STEP 15: COPY . ${SRC_DIR}
    STEP 16: RUN make cephcsi
    cephcsi image settings: quay.io/cephcsi/cephcsi version canary
    make: git: Command not found
    make: git: Command not found
    if [ ! -d ./vendor ]; then (go mod tidy && go mod vendor); fi
    make: git: Command not found
    ...
    STEP 23: COMMIT
    Getting image source signatures
    ...
    Writing manifest to image destination
    Storing signatures
    --> 239b19c4049

git is used to detect the current commit, and store it in the binary
that is built. Without the commit, the "Git Commit:" in the output is
empty, making it impossible to get the exact version:

    $ podman run --rm 239b19c4049 --version
    Cephcsi Version: canary
    Git Commit:
    Go Version: go1.15
    Compiler: gc
    Platform: linux/amd64
    Kernel: 5.8.4-200.fc32.x86_64

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-27 21:46:38 +00:00
Humble Chirammal
e154029e6d deploy: update csi-attacher to v2.2.0 from v2.1.0
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-10-13 12:34:42 +00:00
Humble Chirammal
bae289ea0a deploy: update sidecar repo paths to new image repository
The image repository has been migrated to k8s.gcr.io/sig-storage from
quay.io/k8scsi.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-10-13 12:34:42 +00:00
Nico Berlee
6a5f1380b0 deploy: add pod anti-affinity for provisioner deployments
The added anti-affinity rules prevent provisioner operators from scheduling on
the same nodes. The kubernetes scheduler will spread the pods across nodes to
improve availability during node failures.

Signed-off-by: Nico Berlee <nico.berlee@on2it.net>
2020-09-29 09:29:58 +00:00