Commit Graph

4367 Commits

Author SHA1 Message Date
Marcel Lauhoff
8d38107fd6 e2e: add basic PVC Ceph FS fscrypt tests
Test storage class, pvc and app bind of an fscrypt encrypted Ceph FS
with secrets metadata, vault, vault tokens and vault tenant KMS.

Tests are based on the RBD block/file encryption tests.

Signed-off-by: Marcel Lauhoff <marcel.lauhoff@suse.com>
2022-11-23 12:21:02 +00:00
Marcel Lauhoff
638f77a95c e2e: Deploy vault as part of the Ceph FS suite
Always deploy Vault as part of the the Ceph FS test suite.

Required by:
 - fscrypt tests using any vault KMS type.
 - Configuration in deploy/cephfs/*.yaml via the
   ceph-csi-encryption-kms-config config map created during deployVault()

Signed-off-by: Marcel Lauhoff <marcel.lauhoff@suse.com>
2022-11-23 12:21:02 +00:00
Marcel Lauhoff
42744213f6 e2e: Add test-cephfs-fscrypt flag
Add flag to default disable Ceph FS fscrypt tests, as they require a
custom minikube ISO

Signed-off-by: Marcel Lauhoff <marcel.lauhoff@suse.com>
2022-11-23 12:21:02 +00:00
Marcel Lauhoff
2ade867211 e2e: Add Ceph FS fscrypt validation helper
Add e2e helper to verify encrypted Ceph FS. Verify file's
ceph.fscrypt.auth attribute and KMS password creation / removal.

Signed-off-by: Marcel Lauhoff <marcel.lauhoff@suse.com>
2022-11-23 12:21:02 +00:00
Marcel Lauhoff
cd42ad67b2 examples: Ceph FS fscrypt / KMS additions
Add encryption configuration to Ceph FS examples

Signed-off-by: Marcel Lauhoff <marcel.lauhoff@suse.com>
2022-11-23 12:21:02 +00:00
Marcel Lauhoff
0e66c3211a deploy: Add KMS configuration to Ceph FS
Adds necessary KMS configuration based on the RBD configuration to use
Ceph FS with fscrypt

Signed-off-by: Marcel Lauhoff <marcel.lauhoff@suse.com>
2022-11-23 12:21:02 +00:00
Marcel Lauhoff
4788d279a5 cephfs: fscrypt encryption support
Add Ceph FS fscrypt support, similar to the RBD/ext4 fscrypt
integration. Supports encrypted PVCs, snapshots and clones.

Requires kernel and Ceph MDS support that is currently not in any
stable release.

Signed-off-by: Marcel Lauhoff <marcel.lauhoff@suse.com>
2022-11-23 12:21:02 +00:00
Madhu Rajanna
28f51aaaf7 e2e: add snapshot count validation
Add snapshot count validation for cephfs
and nfs to avoid resource leak.

fixes: #3224

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-11-21 12:16:10 +00:00
Madhu Rajanna
c5a6d11a8f e2e: correct int format
use %d when formatting the int
value.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-11-21 12:16:10 +00:00
dependabot[bot]
0f0957164e rebase: bump github.com/aws/aws-sdk-go-v2/service/sts
Bumps [github.com/aws/aws-sdk-go-v2/service/sts](https://github.com/aws/aws-sdk-go-v2) from 1.17.1 to 1.17.3.
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/v1.17.1...config/v1.17.3)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/service/sts
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-11-17 13:36:07 +00:00
dependabot[bot]
f0cc5a0ef8 rebase: bump actions/dependency-review-action from 2 to 3
Bumps [actions/dependency-review-action](https://github.com/actions/dependency-review-action) from 2 to 3.
- [Release notes](https://github.com/actions/dependency-review-action/releases)
- [Commits](https://github.com/actions/dependency-review-action/compare/v2...v3)

---
updated-dependencies:
- dependency-name: actions/dependency-review-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-11-17 13:00:22 +00:00
Humble Chirammal
b839c3aa63 deploy: remove snapshot v1beta1 references from manifests
This commit remove the v1beta1 snapshot references as its
no longer valid or to be concerned about.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-11-17 10:05:01 +00:00
riya-singhal31
539686329f ci: fix mdl related failures
This commit address the issue-
https://github.com/ceph/ceph-csi/issues/3448.

Signed-off-by: riya-singhal31 <rsinghal@redhat.com>
2022-11-17 08:25:10 +00:00
Humble Chirammal
d721ed6c5c build: fix CVEs in the image
This commit update dependencies which is required to fix below CVEs.

CVE-2022-27664
CVE-2022-27191

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-11-16 15:16:16 +00:00
dependabot[bot]
4e9047dcbd rebase: bump github.com/pkg/xattr from 0.4.7 to 0.4.9
Bumps [github.com/pkg/xattr](https://github.com/pkg/xattr) from 0.4.7 to 0.4.9.
- [Release notes](https://github.com/pkg/xattr/releases)
- [Commits](https://github.com/pkg/xattr/compare/v0.4.7...v0.4.9)

---
updated-dependencies:
- dependency-name: github.com/pkg/xattr
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-11-16 10:26:31 +00:00
Niels de Vos
93d32c49c8 build: fix ShellCheck issue in scripts/test-go.sh
With the updated Fedora 37 container-image, a new version of ShellCheck
gets installed. This version is a little more strict and complains about
the array expansion in `scripts/test-go.sh`.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-11-16 08:01:46 +00:00
Niels de Vos
774beef838 ci: install openssl for Fedora 37 testing image
GitHub Workflows fail installing Helm if the `openssl` package is not
available. Fedora 36 installs `openssl` by default, Fedora 37 does not.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-11-16 08:01:46 +00:00
Humble Chirammal
ff18fb1def build: add dnf update and add switch --nodocs to install command
this commit update the packages and then do installation of the
packages in docker build process.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-11-15 15:32:57 +00:00
Humble Chirammal
b134bf7eda build: update golang version to 1.18.8
the latest 1.18 version of go binary is 1.18.8 and this commit
update the package to the latest.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-11-15 15:32:57 +00:00
Humble Chirammal
f9530e961b build: update packages in api directory to the latest
this commit update the packages in API directory and also
update the mention of go version in the same.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-11-15 09:51:29 +00:00
Humble Chirammal
95dac056f2 ci: fix formatting for the interval in dependabot configuration
as per the documentation `""` has to be mentioned for the schedule
interval value field. This commit ensures it and make it consistent.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-11-15 03:34:37 +00:00
Humble Chirammal
a2215683e1 ci: package dependencies in actions/retest
these dependencies were not updated and this commit update the
same.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-11-15 03:34:37 +00:00
Humble Chirammal
71c4ae542c rebase: remove protobuf dependency locking
this commit remove the protobuf dependency locking in the module
description.

Also, ptypes.TimestampProto is deprecated and this commit
make use of the timestamppb.New() for the construction.

ParseTime() function has been removed and callers adjusted to the
same.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-11-15 00:10:46 +00:00
Humble Chirammal
d772fc098c rebase: update to go-ceph v1.18.0
this commit make use of latest go-ceph version
https://github.com/ceph/go-ceph/releases/tag/v0.18.0

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-11-14 20:46:41 +00:00
Humble Chirammal
ea806bfa73 docs: update/correct development guide
the pre-commit version in the developement guide was too outdated
which has been updated and this commit also addressed a typo.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-11-14 20:46:41 +00:00
Madhu Rajanna
d12400aa9c rbd: unset metadata if setmetadata is false
We need to unset the metadata on the clone
and restore PVC if the parent PVC was created
when setmetadata was set to true and it was
set to false when restore and clone pvc was
created.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-11-14 14:41:36 +00:00
Humble Chirammal
ad36f13e92 build: the go setup has been configured to 1.18 for retest build
this commit make use of 1.18 version of go for building
actions/retest code.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-11-12 07:16:21 +00:00
Rakshith R
eb21d75ef7 rbd: ignore stdErr for ceph osd blocklist when there is no error
`ceph osd blocklist range add/rm <ip>` cmd is outputting
"blocklisting cidr:10.1.114.75:0/32 until 202..." messages
incorrectly into stdErr. This commit ignores stdErr when err
is nil.

Signed-off-by: Rakshith R <rar@redhat.com>
2022-11-12 04:20:14 +00:00
Humble Chirammal
e2832fde5b deploy: add fsgrouppolicy to the driver yaml
this fsgrouppolicy setting was missing in api/deploy/* which
caused the yamlgen to not pickup this. this commit address the
same.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-11-11 16:41:24 +00:00
Humble Chirammal
b258628b05 helm: get rid of storage group enablement based on the version
deploy: remove beta storage group mention from csidriver yaml

the kubernetes version based enablement of storage api group
enablement is no longer requried and its already on v1 for
supported kubernetes versions.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-11-11 16:41:24 +00:00
Humble Chirammal
eff8a9b3be helm: fuse_set_user_groups need not be part of the config
this setting in ceph.conf is no longer required and this commit
remove it from the chart.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-11-11 16:41:24 +00:00
Niels de Vos
38c4832665 ci: do not leave testing comments on merged PRs
A PR with status `github.event.pull_request.merged == true` does not
need to be tested again.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-11-11 11:35:05 +00:00
Humble Chirammal
c9ccbf29bb rebase: update to latest snapshotter
this commit update the snapshotter client to v6.1.0

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-11-11 09:25:57 +00:00
dependabot[bot]
8b078f1a11 rebase: bump golang.org/x/sys from 0.1.0 to 0.2.0
Bumps [golang.org/x/sys](https://github.com/golang/sys) from 0.1.0 to 0.2.0.
- [Release notes](https://github.com/golang/sys/releases)
- [Commits](https://github.com/golang/sys/compare/v0.1.0...v0.2.0)

---
updated-dependencies:
- dependency-name: golang.org/x/sys
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-11-10 18:23:31 +00:00
Niels de Vos
9a61d9cbfc ci: do not immediately remove the ok-to-test label
It seems that Mergify removes the `ok-to-test` label as soon as someone
adds it. We don't want that, as it can trigger more CI runs than needed.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-11-10 11:40:28 +01:00
Humble Chirammal
24fb43d30d doc: correct csi configuration references in the deployment
this commit correct the csi configuration documentation in
rbd deployment doc.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-11-10 10:16:37 +00:00
Humble Chirammal
0c06233f40 doc: remove uid/gid mgr issue from the deployment doc
this issue has been already fixed long back and available with
ceph backports. We no longer require this mention in the docs.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-11-10 10:16:37 +00:00
Humble Chirammal
6009b7c7cf doc: lift the minimum supported version of ceph to v15.0.0
at present the supported ceph cluster version has been listed with
v14.x.x version of Ceph. However upstream Ceph has dropped its
support for 14.x version and minimum supported release is v15.x

Ref # https://docs.ceph.com/en/quincy/releases/index.html

this commit update the ceph cluster version mention to v15.0.0

Additional Note:
The code should work even for last archived release but we dont
need that mention anymore in the readme.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-11-10 10:04:00 +00:00
Niels de Vos
2518c39956 ci: automatically remove ok-to-test and set it when queuing a PR
The `ok-to-test` label currently needs to be removed and re-added when a
PR is rebased for re-queuing.

It should be possible to automate this, by removing the `ok-to-test`
label when a PR gets queued. It can automatically be added again when
the PR has embarked the merge train.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-11-10 15:11:28 +05:30
Humble Chirammal
628263e9b1 e2e: use CurrentSpecReport for failed spec run
CurrentGinkgoTestDescription() has been deprecated in favor of
CurrentSpecReport and this commit address the same.

Ref#https://github.com/onsi/ginkgo/blob/master/deprecated_dsl.go#L53

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-11-10 09:30:38 +00:00
Madhu Rajanna
cf237d1f7e ci: update minikube to v1.28.0
update minikube to latest available
release.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-11-09 18:50:14 +00:00
Humble Chirammal
d70b594946 rbd: remove false error check in getDeviceSize
this removed err condition will be always false as error
is always nil.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-11-09 15:35:45 +00:00
Humble Chirammal
1695240f54 build: remove travis CI scripts for functest and helm
considering we have moved out of travis and replaced github
action, this commit remove the functional test and helmtest
scripts.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-11-09 13:12:57 +00:00
dependabot[bot]
c65a4e1d8a rebase: bump github.com/aws/aws-sdk-go from 1.44.127 to 1.44.132
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.127 to 1.44.132.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.44.127...v1.44.132)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-11-08 12:41:48 +00:00
Madhu Rajanna
03480d2927 rebase: update csi to v1.7.0
Update csi to latest v1.7.0 release
https://github.com/container-storage-interface
/spec/releases/tag/v1.7.0

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-11-08 11:55:16 +00:00
dependabot[bot]
076816d26e rebase: bump google.golang.org/grpc from 1.49.0 to 1.50.1
Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.49.0 to 1.50.1.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.49.0...v1.50.1)

---
updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-11-08 10:00:37 +00:00
dependabot[bot]
fe13fff9fa rebase: bump github.com/IBM/keyprotect-go-client from 0.8.1 to 0.9.0
Bumps [github.com/IBM/keyprotect-go-client](https://github.com/IBM/keyprotect-go-client) from 0.8.1 to 0.9.0.
- [Release notes](https://github.com/IBM/keyprotect-go-client/releases)
- [Commits](https://github.com/IBM/keyprotect-go-client/compare/v0.8.1...v0.9.0)

---
updated-dependencies:
- dependency-name: github.com/IBM/keyprotect-go-client
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-11-08 08:20:05 +00:00
Rakshith R
8650538b78 rbd: setup encryption if rbdVol exits during CreateVol
This commit adds code to setup encryption on a rbdVol
being repaired in a followup CreateVolume request.
This is fixes a bug wherein encryption metadata may not
have been set in previous request due to container restart.

Fixes: #3402

Signed-off-by: Rakshith R <rar@redhat.com>
2022-11-07 12:49:18 +00:00
Madhu Rajanna
07e9dede2c rbd: check volume details from original volumeID
Checking volume details for the existing volumeID
first. if details like OMAP, RBD Image, Pool doesnot
exists try to use clusterIDMapping to look for the
correct informations.

fixes: #2929

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-11-04 06:32:05 +00:00
Humble Chirammal
6bba64c872 rebase: update node driver registar to latest version
v2.6.0 is the latest version of the same and this commit
update it.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-11-03 14:31:26 +00:00