Commit Graph

3743 Commits

Author SHA1 Message Date
Humble Chirammal
66e7f3525f cleanup: remove unimplemented controller expand,snapshot RPCs
These RPCs ( controller expand, create and delete snapshots) are
no longer unimplmented and we dont have to declare these as with
`unimplemented` states. This commit remove the same.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-03-16 15:27:48 +00:00
Niels de Vos
1b16db4ff6 doc: proposal for providing NFS-exports
The design for NFS-provising of volumes explains how the existing NFS
CSI Node-Plugin can be used, and what the steps are to create the
NFS-export with Ceph Mgr.

Updates: #2913
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-03-16 11:36:18 +00:00
Rakshith R
23145bc1f7 cleanup: fix csi-kms-connection-details.yaml ibmkeyprotect-test ex
The last option in `ibmkeyprotect-test` example had a extra `,`
at the end, which caused parsing to fail. This commit removes
the same.

Signed-off-by: Rakshith R <rar@redhat.com>
2022-03-16 07:29:56 +00:00
Rakshith R
4f0bb2315b rbd: add aws-sts-metdata encryption type
With Amazon STS and kubernetes cluster is configured with
OIDC identity provider, credentials to access Amazon KMS
can be fetched using oidc-token(serviceaccount token).
Each tenant/namespace needs to create a secret with aws region,
role and CMK ARN.
Ceph-CSI will assume the given role with oidc token and access
aws KMS, with given CMK to encrypt/decrypt DEK which will stored
in the image metdata.

Refer: https://docs.aws.amazon.com/STS/latest/APIReference/welcome.html
Resolves: #2879

Signed-off-by: Rakshith R <rar@redhat.com>
2022-03-16 07:29:56 +00:00
dependabot[bot]
13dcc89ac8 rebase: bump google.golang.org/grpc from 1.44.0 to 1.45.0
Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.44.0 to 1.45.0.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.44.0...v1.45.0)

---
updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-03-15 12:19:19 +00:00
dependabot[bot]
c3e35f8849 rebase: bump github.com/aws/aws-sdk-go from 1.43.13 to 1.43.18
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.43.13 to 1.43.18.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Changelog](https://github.com/aws/aws-sdk-go/blob/main/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.43.13...v1.43.18)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-03-15 06:25:30 +00:00
Prasanna Kumar Kalever
3eb0fa5e21 rbd: fix parsing mapOptions
Currently, we support

mapOption: "krbd:v1,v2,v3;nbd:v1,v2,v3"

- By omitting `krbd:` or `nbd:`, the option(s) apply to
  rbdDefaultMounter which is krbd.
- A user can _override_ the options for a mounter by specifying `krbd:`
  or `nbd:`.
  mapOption: "v1,v2,v3;nbd:v1,v2,v3"
  is effectively the same as the 1st example.
- Sections are split by `;`.
- If users want to specify common options for both `krbd` and `nbd`,
  they should mention them twice.

But in case if the krbd or nbd specifc options contian `:` within them,
then the parsing is failing now.

E0301 10:19:13.615111 7348 utils.go:200] ID: 63 Req-ID:
0001-0009-rook-ceph-0000000000000001-fd37c41b-9948-11ec-ad32-0242ac110004
GRPC error: badly formatted map/unmap options:
"krbd:read_from_replica=localize,crush_location=zone:zone1;"

This patch fix the above case where the options itself contain `:`
delimitor
ex: krbd:v1,v2,v3=v31:v32;nbd:v1,v2,v3"

Please note, if you are using such options which contain `:` delimiter,
then it is mandatory to specify the mounter-type.

Fixes: #2910
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2022-03-14 15:21:25 +00:00
dependabot[bot]
c435c7bf21 rebase: bump github.com/aws/aws-sdk-go from 1.43.8 to 1.43.13
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.43.8 to 1.43.13.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Changelog](https://github.com/aws/aws-sdk-go/blob/main/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.43.8...v1.43.13)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-03-14 11:20:52 +00:00
dependabot[bot]
4e98cff11e rebase: bump github.com/kubernetes-csi/csi-lib-utils
Bumps [github.com/kubernetes-csi/csi-lib-utils](https://github.com/kubernetes-csi/csi-lib-utils) from 0.10.0 to 0.11.0.
- [Release notes](https://github.com/kubernetes-csi/csi-lib-utils/releases)
- [Commits](https://github.com/kubernetes-csi/csi-lib-utils/compare/v0.10.0...v0.11.0)

---
updated-dependencies:
- dependency-name: github.com/kubernetes-csi/csi-lib-utils
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-03-11 10:59:20 +00:00
Madhu Rajanna
78ec859dc6 cleanup: remove unwanted print
Removing unwanted print from the code

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-03-11 05:40:32 +00:00
Humble Chirammal
7c966b5a39 doc: correct static pvc mention for cephfs
it was wrongly mentioned RBD PVC in cephfs static pvc section.
This commit removes the same.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-03-10 10:32:36 +00:00
Robert Vasek
78697e4f7c doc: added docs for ceph-fuse mount corruption
Signed-off-by: Robert Vasek <robert.vasek@cern.ch>
2022-03-10 06:05:52 +00:00
Robert Vasek
548a752ff0 e2e: added test for verifying that ceph-fuse recovery works for new pods
Signed-off-by: Robert Vasek <robert.vasek@cern.ch>
2022-03-10 06:05:52 +00:00
Robert Vasek
80dda7cc30 cephfs: detect corrupt ceph-fuse mounts and try to remount
Mounts managed by ceph-fuse may get corrupted by e.g. the ceph-fuse process
exiting abruptly, or its parent container being terminated, taking down its
child processes with it.

This commit adds checks to NodeStageVolume and NodePublishVolume procedures
to detect whether a mountpoint in staging_target_path and/or target_path is
corrupted, and remount is performed if corruption is detected.

Signed-off-by: Robert Vasek <robert.vasek@cern.ch>
2022-03-10 06:05:52 +00:00
Robert Vasek
aa6297e164 cleanup: refactor helper functions in nodeserver.go
Refactored a couple of helper functions for easier resue.

* Code for building store.VolumeOptions is factored out into a separate function.

* Changed args of getCredentailsForVolume() and NodeServer.mount() so that
  instead of passing in whole csi.NodeStageVolumeRequest, only necessary
  properties are passed explicitly. This is to allow these functions to be
  called outside of NodeStageVolume() where NodeStageVolumeRequest is not
  available.

Signed-off-by: Robert Vasek <robert.vasek@cern.ch>
2022-03-10 06:05:52 +00:00
Rakshith R
dec74a534f doc: add step to create ceph-conf cm to deploy plugin
Ceph-csi requires `ceph-config` configmap when deploying.
Add this step to `examples/README.md`.

Fixes: #2918

Signed-off-by: Rakshith R <rar@redhat.com>
2022-03-09 06:39:57 +00:00
Benjamin Guillon
d236968bf9 helm: remove kube version semver check for CSI cephfs resizer component
It was decided that latest ceph CSI versions would drop support for
older Kubernetes versions, making this check useless. So it was removed.

Removing this version check allows for the deployment of the CephFS
resizer component when using the helm chart on non vanilla kubernetes
clusters whose API server version are in the form of `1.x.y-abc+def-ghi`.

Signed-off-by: Benjamin Guillon <benjamin.guillon@cc.in2p3.fr>
2022-03-09 06:07:49 +00:00
Rakshith R
3a64ee48c3 rbd: return unimplemented error for block-mode reclaimspace req
blkdiscard cmd discards all data on the block device which
is not desired. Hence, return unimplemented code if the
volume access mode is block.

Signed-off-by: Rakshith R <rar@redhat.com>
2022-03-03 19:00:49 +00:00
Madhu Rajanna
dd96024620 doc: remove mention of image feature dependency
avoid specifying the image feature dependencies
and add a link to rbd official document for
reference to the image feature dependencies.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-03-02 09:08:33 +00:00
Niels de Vos
1f012004a6 util: configure tenants vaultAuthNamespace if not set
When a tenant provides a configuration that includes the
`vaultNamespace` option, the `vaultAuthNamespace` option is still taken
from the global configuration. This is not wanted in all cases, as the
`vaultAuthNamespace` option defauls to the `vaultNamespace` option which
the tenant may want to override as well.

The following behaviour is now better defined:

1. no `vaultAuthNamespace` in the global configuration:
   A tenant can override the `vaultNamespace` option and that will also
   set the `vaultAuthNamespace` option to the same value.

2. `vaultAuthNamespace` and `vaultNamespace` in the global configuration:
   When both options are set to different values in the global
   configuration, the tenant `vaultNamespace` option will not override
   the global `vaultAuthNamespace` option. The tenant can configure
   `vaultAuthNamespace` with a different value if required.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-03-02 08:36:33 +00:00
dependabot[bot]
b1af5f63b5 rebase: bump github.com/hashicorp/vault/api from 1.3.1 to 1.4.1
Bumps [github.com/hashicorp/vault/api](https://github.com/hashicorp/vault) from 1.3.1 to 1.4.1.
- [Release notes](https://github.com/hashicorp/vault/releases)
- [Changelog](https://github.com/hashicorp/vault/blob/main/CHANGELOG.md)
- [Commits](https://github.com/hashicorp/vault/compare/v1.3.1...v1.4.1)

---
updated-dependencies:
- dependency-name: github.com/hashicorp/vault/api
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-03-01 10:37:45 +00:00
dependabot[bot]
9a9c69cba2 rebase: bump github.com/aws/aws-sdk-go from 1.43.3 to 1.43.8
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.43.3 to 1.43.8.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Changelog](https://github.com/aws/aws-sdk-go/blob/main/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.43.3...v1.43.8)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-03-01 08:11:06 +00:00
Madhu Rajanna
d5c98f81a2 rbd: make image features as optional parameter
Makes the rbd images features in the storageclass
as optional so that default image features of librbd
can be used. and also kept the option to user
to specify the image features in the storageclass.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-02-28 13:10:03 +00:00
Madhu Rajanna
fb3835691f rbd: add support for deep-flatten image feature
as deep-flatten is long supported in ceph and its
enabled by default in the librbd, providing an option
to enable it in cephcsi for the rbd images we are
creating.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-02-28 13:10:03 +00:00
Humble Chirammal
eb40fbcb18 doc: remove deprecated alpha1 reference of snapshot objects
alpha1 version is deprecated and this commit correct the
example doc and make it GAd version

Additional note: we have already moved the artifacts to v1 version.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-02-25 08:42:46 +00:00
Robert Vasek
e9c6fab287 e2e: loadAppDeployment now sets ImagePullPolicy to PullIfNotPresent
Fixes: #2889

Signed-off-by: Robert Vasek <robert.vasek@cern.ch>
2022-02-24 15:38:24 +00:00
dependabot[bot]
ac6bfeb6ef rebase: bump k8s.io/kubernetes from 1.23.3 to 1.23.4
Bumps [k8s.io/kubernetes](https://github.com/kubernetes/kubernetes) from 1.23.3 to 1.23.4.
- [Release notes](https://github.com/kubernetes/kubernetes/releases)
- [Commits](https://github.com/kubernetes/kubernetes/compare/v1.23.3...v1.23.4)

---
updated-dependencies:
- dependency-name: k8s.io/kubernetes
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-02-23 14:47:13 +00:00
Madhu Rajanna
e9802c4940 cephfs: refactor cephfs core functions
This commits refactors the cephfs core
functions with interfaces. This helps in
better code structuring and writing the
unit test cases.

update #852

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-02-22 20:39:23 +00:00
dependabot[bot]
f19ca4a473 rebase: bump github.com/aws/aws-sdk-go from 1.42.53 to 1.43.3
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.42.53 to 1.43.3.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Changelog](https://github.com/aws/aws-sdk-go/blob/main/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.42.53...v1.43.3)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-02-22 07:09:48 +00:00
dependabot[bot]
cd83c7be48 rebase: bump github.com/ceph/go-ceph from 0.13.0 to 0.14.0
Bumps [github.com/ceph/go-ceph](https://github.com/ceph/go-ceph) from 0.13.0 to 0.14.0.
- [Release notes](https://github.com/ceph/go-ceph/releases)
- [Changelog](https://github.com/ceph/go-ceph/blob/master/docs/release-process.md)
- [Commits](https://github.com/ceph/go-ceph/compare/v0.13.0...v0.14.0)

---
updated-dependencies:
- dependency-name: github.com/ceph/go-ceph
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-02-22 04:54:50 +00:00
Madhu Rajanna
3922cfc860 e2e: consider not found error in deployment check
it might need sometime for the deployment to
get created, consider the NotFound as a valid
error and retry again.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-02-21 12:18:39 +00:00
Silvio Gissi
9c50e255fb helm: make ceph.conf ConfigMap name configurable
ConfigMap name was hardcoded and led to conflicts. Fixes #2858.

Signed-off-by: Silvio Gissi <silvio@gissilabs.com>
2022-02-21 07:25:22 +00:00
Francesco Astegiano
4235178f7c helm: Add selinuxMount flag to enable/disable /etc/selinux host mount
Add selinuxMount flag to enable/disable /etc/selinux host mount inside pods
to support selinux-enabled filesystems

Signed-off-by: Francesco Astegiano <francesco.astegiano@gmail.com>
2022-02-16 12:48:00 +00:00
Niels de Vos
ea89b26f65 e2e: pass --ignore-not-found=true when deleting resources
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-02-15 16:21:07 +00:00
Niels de Vos
a669258688 cleanup: remove unused global variable in e2e
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-02-15 16:21:07 +00:00
Niels de Vos
26696b4979 e2e: retry deleting Rook artifacts in case of error
On occasion deploying CephFS components fail due to errors like these:

    failed to delete provisioner rbac .../csi-provisioner-rbac.yaml

By using the deleteResource() helper, an retry is done in case of a
failure.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-02-15 16:21:07 +00:00
Niels de Vos
8a0377ef02 e2e: retry creating StorageClass in case of error
There have been errors while CephFS tests were running, like:

    failed to create storageclass: etcdserver: request timed out

When retrying to create the StorageClass, the e2e tests are expected to
continue and (hopefully) succeed.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-02-15 16:21:07 +00:00
Deividas Burškaitis
91c22f521b helm: add port sections to helm templates
to show what ports containers are exposing add port sections to nodeplugin
and provisioner helm templates

Signed-off-by: Deividas Burškaitis <deividas.burskaitis@oxylabs.io>
2022-02-15 10:06:26 +00:00
dependabot[bot]
7064642dee rebase: bump github.com/aws/aws-sdk-go from 1.42.48 to 1.42.53
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.42.48 to 1.42.53.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Changelog](https://github.com/aws/aws-sdk-go/blob/main/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.42.48...v1.42.53)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-02-15 06:57:12 +00:00
Madhu Rajanna
46378f3bfc rbd: log stderror when running modprobe
logging the error is not user-friendly and
it contains system error message. Log the
stderr which is user-friendly error message
for identifying the problem.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-02-14 15:03:31 +00:00
Madhu Rajanna
0311eb5f44 helm: remove namespace from storageclass yaml
removes namespace from non-namespaced storageclass
object.

fixes: #2714

Replacement for #2715 as we didnt receive any update
and PR is already closed.

Co-authored-by: jhrcz-ls
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-02-11 12:32:58 +00:00
Sébastien BERNARD
ee8fb3f05f rbd: Fix dataPool in createVolumeResponse
Return the dataPool used to create the image instead of the default one
provided by the createVolumeRequest.
In case of topologyConstrainedDataPools, they may differ.
Don't add datapool if it's not present

Signed-off-by: Sébastien Bernard <sebastien.bernard@sfr.com>
2022-02-10 11:44:22 +00:00
Humble Chirammal
8f6a7da538 cephfs: dont set explicit permissions on the volume
At present we are node staging with worldwide permissions which is
not correct. We should allow the CO to take care of it and make
the decision. This commit also remove `fuseMountOptions` and
`KernelMountOptions` as they are no longer needed

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-02-09 17:30:29 +00:00
dependabot[bot]
c0e70ca8cb rebase: bump k8s.io/kubernetes from 1.23.2 to 1.23.3
Bumps [k8s.io/kubernetes](https://github.com/kubernetes/kubernetes) from 1.23.2 to 1.23.3.
- [Release notes](https://github.com/kubernetes/kubernetes/releases)
- [Commits](https://github.com/kubernetes/kubernetes/compare/v1.23.2...v1.23.3)

---
updated-dependencies:
- dependency-name: k8s.io/kubernetes
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-02-09 15:16:21 +00:00
dependabot[bot]
266e5b3b80 rebase: bump github.com/onsi/gomega from 1.18.0 to 1.18.1
Bumps [github.com/onsi/gomega](https://github.com/onsi/gomega) from 1.18.0 to 1.18.1.
- [Release notes](https://github.com/onsi/gomega/releases)
- [Changelog](https://github.com/onsi/gomega/blob/master/CHANGELOG.md)
- [Commits](https://github.com/onsi/gomega/compare/v1.18.0...v1.18.1)

---
updated-dependencies:
- dependency-name: github.com/onsi/gomega
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-02-09 07:48:51 +00:00
dependabot[bot]
d1d4d92131 rebase: bump github.com/aws/aws-sdk-go from 1.42.40 to 1.42.48
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.42.40 to 1.42.48.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Changelog](https://github.com/aws/aws-sdk-go/blob/main/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.42.40...v1.42.48)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-02-09 05:33:25 +00:00
Madhu Rajanna
2943555904 cephfs: fix omap deletion in DeleteSnapshot
the omap is stored with the requested
snapshot name not with the subvolume
snapshotname. This fix uses the correct
snapshot request name to cleanup the omap
once the subvolume snapshot is deleted.

fixes: #2832

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-02-08 20:37:53 +00:00
Humble Chirammal
ad6a3d7575 rbd: remove kp-metadata register functions of HPCS/Key Protect
This commit removes `kp-metadata` registration from existing HPCS
or Key Protect code as per the plan.

Fix #2816

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-02-08 18:27:03 +00:00
dependabot[bot]
7654a56367 rebase: bump google.golang.org/grpc from 1.43.0 to 1.44.0
Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.43.0 to 1.44.0.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.43.0...v1.44.0)

---
updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-02-08 14:54:05 +00:00
Humble Chirammal
1c3baa0722 rbd: add AAD(additionalAuthData) while unwrapping the DEK
As we are using optional additional auth data while wrapping
the DEK, we have to send the same additionally while unwrapping.

Error:
```
 failed to unwrap the DEK: kp.Error: ..(INVALID_FIELD_ERR)',
 reasons='[INVALID_FIELD_ERR: The field `ciphertext` must be: the
 original base64 encoded ciphertext from the wrap operation
```

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-02-08 03:06:30 +00:00