Commit Graph

4032 Commits

Author SHA1 Message Date
Rakshith R
5983d5730a deploy: add support for nfs snapshot
This commit add external-snapshotter container and
required rbac to support nfs snaphots.
Example volumesnaphotclass, volumesnapshot,
pvc-restore and pod-restore yamls are also added.

Signed-off-by: Rakshith R <rar@redhat.com>
2022-05-24 18:13:02 +00:00
Rakshith R
24515b509f nfs: add support for create & delete snapshot
This commits adds support for creation and
deletion of nfs snapshots based on cephfs.

Signed-off-by: Rakshith R <rar@redhat.com>
2022-05-24 18:13:02 +00:00
Marcel Lauhoff
b7ec0b2d66 doc: add proposal for CephFS fscrypt integration
Add proposal document covering key management integration
of Ceph CSI and https://github.com/google/fscrypt

Updates: #1563
Signed-off-by: Marcel Lauhoff <marcel.lauhoff@suse.com>
2022-05-24 15:02:08 +00:00
Naveen
60281fab40 ci: included dependency review
Dependency Review GitHub Action in your repository to enforce dependency
reviews on your pull requests.

Signed-off-by: Naveen <172697+naveensrinivasan@users.noreply.github.com>
2022-05-24 06:48:22 +00:00
dependabot[bot]
386cd7f20e rebase: bump github.com/aws/aws-sdk-go from 1.44.16 to 1.44.20
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.16 to 1.44.20.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Changelog](https://github.com/aws/aws-sdk-go/blob/main/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.44.16...v1.44.20)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-05-24 06:07:44 +00:00
Humble Chirammal
ba93a2b037 cephfs: adjust RBAC for CephFS provisioner role
CephFS CSI driver does not need permissions on Node,ConfigMap objects.
This PR also adjust the update->Patch for a couple of RBAC.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-05-23 21:52:33 +00:00
Humble Chirammal
7e1ed75cfb doc: remove topology mention from cephfs documentation
as we are removing the topology configuration from the deployment
this commit remove it from the documentation too.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-05-23 10:57:56 +00:00
Humble Chirammal
29870cdbcc helm: remove topology RBAC section for cephfs deployment
At present CephFS does not support topology aware provisioning and this
commit remove the same.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-05-23 10:57:56 +00:00
dependabot[bot]
9d3086e211 rebase: bump github.com/aws/aws-sdk-go from 1.44.10 to 1.44.16
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.10 to 1.44.16.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Changelog](https://github.com/aws/aws-sdk-go/blob/main/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.44.10...v1.44.16)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-05-19 09:31:11 +00:00
Rakshith R
952105e551 e2e: remove TODO update nfs node-plugin with fix csi-driver-nfs#319
This commits removes
TODO: update nfs node-plugin that has kubernetes-csi/csi-driver-nfs#319
Since, the nfsplugin image is already updated to v4.0.0.

Signed-off-by: Rakshith R <rar@redhat.com>
2022-05-17 21:56:46 +00:00
dependabot[bot]
eea957b6b3 rebase: bump google.golang.org/grpc from 1.46.0 to 1.46.2
Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.46.0 to 1.46.2.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.46.0...v1.46.2)

---
updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-05-17 17:27:50 +00:00
dependabot[bot]
0b043d3c21 rebase: bump github.com/aws/aws-sdk-go-v2/service/sts
Bumps [github.com/aws/aws-sdk-go-v2/service/sts](https://github.com/aws/aws-sdk-go-v2) from 1.16.4 to 1.16.5.
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/ivs/v1.16.4...service/ivs/v1.16.5)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/service/sts
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-05-17 14:57:22 +00:00
dependabot[bot]
4007283a5d rebase: bump github.com/prometheus/client_golang from 1.12.1 to 1.12.2
Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) from 1.12.1 to 1.12.2.
- [Release notes](https://github.com/prometheus/client_golang/releases)
- [Changelog](https://github.com/prometheus/client_golang/blob/main/CHANGELOG.md)
- [Commits](https://github.com/prometheus/client_golang/compare/v1.12.1...v1.12.2)

---
updated-dependencies:
- dependency-name: github.com/prometheus/client_golang
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-05-17 07:30:35 +00:00
Prasanna Kumar Kalever
6470cf3343 rbd: fix bug handling GetKrbdSupportedFeatures()
continue running rbd driver when /sys/bus/rbd/supported_features file is
missing, do not bailout.

Fixes: #2678
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2022-05-15 15:10:08 +00:00
Prasanna Kumar Kalever
83cc1b0e58 rbd: handle when krbdFeatures is zero
krbdFeatures is set to zero when kernel version < 3.8, i.e. in  case where
/sys/bus/rbd/supported_features is absent and we are unable to prepare
the krbd attributes based on kernel version.

When krbdFeatures is set to zero fallback to NBD only when autofallback
is turned ON.

Fixes: #2678
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2022-05-15 15:10:08 +00:00
Prasanna Kumar Kalever
e53fd87154 rbd: prepare krbd feature attrs if supported_features file is absent
Upstream /sys/bus/rbd/supported_features is part of Linux kernel v4.11.0
Prepare the attributes and use them in case if
/sys/bus/rbd/supported_features is missing.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2022-05-15 15:10:08 +00:00
Prasanna Kumar Kalever
ba6052e896 e2e: workaround fix for omap leaks
2 omap objects are getting leaked in the e2e tests, this change is to
workaround them for now.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2022-05-12 18:47:27 +00:00
Prasanna Kumar Kalever
ea8a3548e6 e2e: validate omap objects per cephfs test case
validate omap count in every testcase right after
validateSubvolumeCount()

Fixes: #2834
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2022-05-12 18:47:27 +00:00
Prasanna Kumar Kalever
dcf51a10b1 e2e: validate omap objects per rbd test case
validate omap count in every testcase right after
validateRBDImageCount()

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2022-05-12 18:47:27 +00:00
Prasanna Kumar Kalever
53f96a8b2a e2e: add utility functions
such as:
listCephFSFileSystems()
getCephFSPoolName()
validateOmapCount()
compareStdoutWithCount()

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2022-05-12 18:47:27 +00:00
Prasanna Kumar Kalever
89a7ada12f e2e: add various testcases to validate unset of metadata
Validate that we:
* Unset the PVC metadata on the rbd image created for the snapshot
* snapshot metadata on CreateVolume from snapshot

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2022-05-12 15:54:09 +00:00
Prasanna Kumar Kalever
27f503c144 rbd: unset parent PVC metadata on CreateVolume From Volume
Unset the parent PVC metadata on the temp clone rbd image

Fixes: #2970
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2022-05-12 15:54:09 +00:00
Prasanna Kumar Kalever
e0f34a6d60 rbd: unset snapshot metadata on CreateVolume From snapshot
Unset the snapshot metadata from the rbd image created from the snapshot

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2022-05-12 15:54:09 +00:00
Prasanna Kumar Kalever
d89c5fb39f rbd: unset PVC metadata on CreateSnapshot
Unset the PVC metadata on the rbd image created for the snapshot

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2022-05-12 15:54:09 +00:00
Prasanna Kumar Kalever
bac33262ae rbd: add unset volume/snapshot metadata utility functions
Added
GetVolumeMetadataKeys()
GetSnaoshotMetadataKeys()
unsetVolumeMetadata() and
unsetSnapshotMetadata()

functions.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2022-05-12 15:54:09 +00:00
Prasanna Kumar Kalever
1fd5277b3c cleanup: simplify setVolumeMetadata and rename it
Move k8s.GetVolumeMetadata() out of setVolumeMetadata() and rename it to
setAllMetadata() so that the same can be used for setting volume and
snapshot metadata.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2022-05-12 15:54:09 +00:00
Niels de Vos
bfef0279a7 e2e: retry getting Pods before executing commands
On occasion the Pods have not been (re)started before they get listed.
This can result in an empty list. It can occur during RBD testing where
Pods are restarted before `uname` is executed. In case the Pods are not
available yet, the test will fail with the "podlist is empty" error.

By adding a retry when the list of Pods is empty, the tests should
become a little more stable.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-05-12 07:43:34 +00:00
Niels de Vos
9f08cb7ea0 e2e: re-order deployment components
Some of the deployment artifacts refer to others (like ServiceAccount in
a Deployment). If the dependencies are not available (yet), there will
be errors reported in the logs. By deploying the components in a more
correct order, fewer errors are reported, making the logs a little
easier to understand.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-05-11 14:58:28 +00:00
dependabot[bot]
7d1c563b97 rebase: bump github.com/aws/aws-sdk-go from 1.44.5 to 1.44.10
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.5 to 1.44.10.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Changelog](https://github.com/aws/aws-sdk-go/blob/main/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.44.5...v1.44.10)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-05-11 07:54:13 +00:00
Niels de Vos
cbd7220b37 doc: mention ExpandVolume support for NFS in the README
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-05-10 17:43:59 +00:00
Niels de Vos
9d7a9144f1 e2e: add ExpandVolume test for NFS-volumes
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-05-10 17:43:59 +00:00
Niels de Vos
891a840aa6 deploy: add resize support to NFS-provisioner deployment
Resizing is handled by the csi-resizer container, which needs to run in
the provisioner Pod. In addition to the container, the StorageClass also
needs to allow volume expansion.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-05-10 17:43:59 +00:00
Niels de Vos
36e51402cb nfs: support ExpandVolume CSI procedure
There is not much the NFS-provisioner needs to do to expand a volume,
everything is handled by the CephFS components.

NFS does not need a resize on the node, so only ControllerExpandVolume
is required.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-05-10 17:43:59 +00:00
Niels de Vos
bbecb81d4f ci: add ci/skip/multi-arch-build label
If the `ci/skip/multi-arch-build` label is set on a PR, the GitHub
Workflow only builds for the local architecture. This makes it possible
to merge PRs faster.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-05-10 06:08:22 +00:00
dependabot[bot]
6d23e750be rebase: bump docker/login-action from 1 to 2
Bumps [docker/login-action](https://github.com/docker/login-action) from 1 to 2.
- [Release notes](https://github.com/docker/login-action/releases)
- [Commits](https://github.com/docker/login-action/compare/v1...v2)

---
updated-dependencies:
- dependency-name: docker/login-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-05-10 05:02:33 +00:00
Niels de Vos
eab2ec42bf e2e: allow kubectlDelete to succeed with NotFound error
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-05-10 00:43:43 +00:00
Niels de Vos
17fea0459d e2e: add -nfs-driver=... option to select provisioner
When testing NFS-provisioning on a cluster that has an NFS-provisioner
and node-plugins deployed with a different driver-name, it is very
useful to have a commandline option to change the name of the
provisioner that is placed in the StorageClass.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-05-10 00:43:43 +00:00
Niels de Vos
9c91710d68 e2e: run NFS-provisioner tests together with CephFS
NFS testing will automatically be enabled when CephFS is enabled. This
makes sure the NFS tests run in the CI where there are different jobs
for CephFS and RBD. With a dedicated testNFS variable, it is still
possible to only run the NFS tests, when both CephFS and RBD are
disabled.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-05-10 00:43:43 +00:00
Niels de Vos
eb70fb9fd4 e2e: add minimal tests for NFS-provisioner
The tests for the NFS-provisioner can be run by passing -deploy-nfs and
-test-nfs as parameters to the `go test` or `e2e.test` command.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-05-10 00:43:43 +00:00
Niels de Vos
085a3bdc73 doc: use .nfs as default pool for NFS-export configs
"nfs-ganesha" is the default pool for older Ceph versions, recent
versions use ".nfs" (which can not be changed in the CephNFS resource).

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-05-10 00:43:43 +00:00
Niels de Vos
1818529a4c deploy: allow hostPort 29653 for NFS node-plugin
This should address the following failure when Pod Security Policies are
enabled:

> FailedCreate: Error creating: pods "csi-nfs-node-" is forbidden:
> PodSecurityPolicy: unable to admit pod: spec.containers[2].hostPort:
> Invalid value: 29653: Host port 29653 is not allowed to be used.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-05-10 00:43:43 +00:00
Niels de Vos
c137af57f3 e2e: add RWOP examples for NFS-provisioning
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-05-10 00:43:43 +00:00
Niels de Vos
782f08e2f0 ci: execCommandInDaemonsetPod should not return unused stdout
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-05-10 00:43:43 +00:00
Niels de Vos
5c1584671e e2e: no need to set Namespace for StorageClass
StorageClasses are cluster-wide, and do not need to have a Namespace
set.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-05-10 00:43:43 +00:00
Madhu Rajanna
70674565df rbd: consider rbd as default mounter if not set
For the default mounter the mounter option
will not be set in the storageclass and as it is
not available in the storageclass same will not
be set in the volume context, Because of this the
mapOptions are getting discarded. If the mounter
is not set assuming it's an rbd mounter.

Note:- If the mounter is not set in the storageclass
we can set it in the volume context explicitly,
Doing this check-in node server to support backward
existing volumes and the check is minimal we are not
altering the volume context.

fixes: #3076

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-05-09 20:00:11 +00:00
Madhu Rajanna
92272d41c5 ci: disable PodSecurity featuregate
PodSecurity featuregate is beta in kubernetes
1.23 and its causing problem for the existing
tests. This PR disables the PodSecurity featuregate
for now and will be enabled later.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-05-09 09:16:12 +00:00
Madhu Rajanna
c4f79d455f rebase: update kubernetes dep to 1.24.0
As kubernetes 1.24.0 is released, updating
kubernetes dependencies to 1.24.0

updates: #3086

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-05-09 09:16:12 +00:00
Niels de Vos
fc1529f268 rebase: bump nfsplugin to v4.0.0
Several bugs have been fixed and new features have been added.

See-also: https://github.com/kubernetes-csi/csi-driver-nfs/releases/tag/v4.0.0
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-05-09 06:34:49 +00:00
Madhu Rajanna
df047ddaaf ci: fix commitlint problem
Still seeing the issue of the commitlint
as below

fatal: unsafe repository
('/go/src/github.com/ceph/ceph-csi'
is owned by someone else)
To add an exception for this directory,
call:

git config --global --add safe.directory \
/go/src/github.com/ceph/ceph-csi
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-05-09 05:57:14 +00:00
Madhu Rajanna
b4ff3884f1 ci: remove set-safe-directory from commitlint
Removed set-safe-directory option from the
commitlint.yaml as its not working as expected.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-05-09 05:57:14 +00:00