Commit Graph

3855 Commits

Author SHA1 Message Date
Marcel Lauhoff
a6a4282493 fscrypt: Unlock: Fetch keys early
Fetch keys from KMS before doing anything else. This will catch KMS
errors before setting up any fscrypt metadata.

Signed-off-by: Marcel Lauhoff <marcel.lauhoff@suse.com>
2022-10-17 17:33:52 +00:00
Marcel Lauhoff
f8faffac89 vendor: vendor fscrypt integration dependencies
Signed-off-by: Marcel Lauhoff <marcel.lauhoff@suse.com>
2022-10-17 17:33:52 +00:00
Marcel Lauhoff
cfea8d7562 fscrypt: fscrypt integration
Integrate google/fscrypt into Ceph CSI KMS and encryption setup. Adds
dependencies to google/fscrypt and pkg/xattr. Be as generic as
possible to support integration with both RBD and Ceph FS.

Add the following public functions:

InitializeNode: per-node initialization steps. Must be called
before Unlock at least once.

Unlock: All steps necessary to unlock an encrypted directory including
setting it up initially.

IsDirectoryUnlocked: Test if directory is really encrypted

Signed-off-by: Marcel Lauhoff <marcel.lauhoff@suse.com>
2022-10-17 17:33:52 +00:00
Marcel Lauhoff
2cf8ecc6c7 journal: Store encryptionType in Config struct
Add encryptionType next to kmsID to support both block and file
encryption.

Signed-off-by: Marcel Lauhoff <marcel.lauhoff@suse.com>
2022-10-17 17:33:52 +00:00
Marcel Lauhoff
ce9fbb3474 rbd: Rename encryption to blockEncryption prep for fscrypt
In preparation of fscrypt support for RBD filesystems, rename block
encryption related function to include the word 'block'. Add struct
fields and IsFileEncrypted.

Signed-off-by: Marcel Lauhoff <marcel.lauhoff@suse.com>
2022-10-17 17:33:52 +00:00
Marcel Lauhoff
624905d60d kms: Add basic GetSecret() test
Add rudimentary test to ensure that we can get a valid passphrase from
the GetSecret() feature

Signed-off-by: Marcel Lauhoff <marcel.lauhoff@suse.com>
2022-10-17 17:33:52 +00:00
Marcel Lauhoff
5df45f1c1b kms: testing: add KMS test dummy registry
Add registry similar to the providers one. This allows testers to
add and use GetKMSTestDummy() to create stripped down provider
instances suitable for use in unit tests.

Signed-off-by: Marcel Lauhoff <marcel.lauhoff@suse.com>
2022-10-17 17:33:52 +00:00
Marcel Lauhoff
cb02a9beb9 kms: Add GetSecret() to metadata KMS
Add GetSecret() to allow direct access to passphrases without KDF and
wrapping by a DEKStore.

This will be used by fscrypt, which has its own KDF and wrapping. It
will allow users to take a k8s secret, for example, and use that
directly as a password in fscrypt.

Signed-off-by: Marcel Lauhoff <marcel.lauhoff@suse.com>
2022-10-17 17:33:52 +00:00
Marcel Lauhoff
0599089de0 util: Add util to fetch encryption type from vol options
Fetch encryption type from vol options. Make fallback type
configurable to support RBD (default block) and Ceph FS (default file)

Signed-off-by: Marcel Lauhoff <marcel.lauhoff@suse.com>
2022-10-17 17:33:52 +00:00
Marcel Lauhoff
fe4821435e util: Make encryption passphrase size a parameter
fscrypt support requires keys longer than 20 bytes. As a preparation,
make the new passphrase length configurable, but default to 20 bytes.

Signed-off-by: Marcel Lauhoff <marcel.lauhoff@suse.com>
2022-10-17 17:33:52 +00:00
Madhu Rajanna
69eb6e40dc rbd: return GRPC error message
The error message return from the GRPC
should be of GRPC error messages only
not the normal go errors. This commits
returns GRPC error if setAllMetadata
fails.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-10-17 15:17:29 +00:00
Madhu Rajanna
01d4a614c3 rbd: delete volume if setallmetadata fails
If any operations fails after the volume creation
we will cleanup the omap objects, but it is missing
if setAllMetadata fails. This commits adds the code
to cleanup the rbd image if metadata operation fails.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-10-17 15:17:29 +00:00
Madhu Rajanna
b40e8894f8 cephfs: use errors.As instead of errors.Is
As we need to compare the error type instead
of the error value we need to use errors.As
to check the API is implemented or not.

fixes: #3347

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-10-17 09:11:45 +00:00
Niels de Vos
b7703faf37 util: make inode metrics optional in FilesystemNodeGetVolumeStats()
CephFS does not have a concept of "free inodes", inodes get allocated
on-demand in the filesystem.

This confuses alerting managers that expect a (high) number of free
inodes, and warnings get produced if the number of free inodes is not
high enough. This causes alerts to always get reported for CephFS.

To prevent the false-positive alerts from happening, the
NodeGetVolumeStats procedure for CephFS (and CephNFS) will not contain
inodes in the reply anymore.

See-also: https://bugzilla.redhat.com/2128263
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-10-13 19:02:47 +00:00
Niels de Vos
386d3ddd6e e2e: disable rbd-nbd tests by default
Because the rbd-nbd tests fail with minikube and the Podman driver,
disable the tests for the time being.

Updates: #3431
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-10-13 10:45:35 +00:00
Niels de Vos
8eaf1d790d e2e: log failures while deleting PVC and PV
There are occasions where deleting a PVC (or PV) never succeeds. The
reported status of the deleted object is sometimes empty, which suggests
that the PVC or PV was, in fact, deleted.

To diagnose the incorrect error checking, include the errors for
retrying in the logs.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-10-13 10:45:35 +00:00
Niels de Vos
fa97875dc9 ci: fail installing Helm if wget is unavailable
In case `wget` is not installed, downloading the Helm release will fail.
The `install-helm.sh` script won't return a fatal error in that case,
and CI jobs continue running in an environment that is not ready.

By adding a check that exist the script with a failure, the CI will now
correctly report a problem when Helm can not be downloaded.

See-also: #3430
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-10-13 10:45:35 +00:00
Niels de Vos
8f915576c4 e2e: wait for deployment before scale down/up
The scale down/up functions fail often with "deployment not found"
errors. Possibly deploying with Podman is slower than deploying in a
minikube VM, and there is a delay for the deployment to become
available.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-10-13 10:45:35 +00:00
Niels de Vos
0cba72485c ci: add support for VM_DRIVER=podman to scripts/minikube.sh
When running on AWE EC2 virtual-machines, we'll use Podman instead of
installing a VM. The "none" driver might work as well, but it requires
additional dependencies to be installed, which may change over time with
new minikube or Kubernetes releases. Hopefully the Podman driver is less
affected with changes in dependencies.

Depends-on: #3419
Closes: #3415
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-10-13 10:45:35 +00:00
Madhu Rajanna
ca12592d5c doc: remove nfs daemonset deletion
As we dont need to delete the nfs daemonset
which was present in 3.6.x release in 3.8.x
release as user will upgrade from 3.6.x to
3.7.x and delete the nfs daemonset.

fixes #3324

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-10-12 14:49:03 +00:00
Madhu Rajanna
71e5b3f922 rbd: remove dummy image workaround
To address the problem that snapshot
schedules are triggered for volumes
that are promoted, a dummy image was
disabled/enabled for replication.
This was done as a workaround, because the
promote operation was not triggering
the schedules for the image being promoted.

The bugs related to the same have been fixed in
RBD mirroring functionality and hence the
workaround #2656 can be removed from the code base.

ceph tracker https://tracker.ceph.com/issues/53914

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-10-10 08:22:10 +00:00
dependabot[bot]
5c8564c69d rebase: bump actions/stale from 5 to 6
Bumps [actions/stale](https://github.com/actions/stale) from 5 to 6.
- [Release notes](https://github.com/actions/stale/releases)
- [Changelog](https://github.com/actions/stale/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/stale/compare/v5...v6)

---
updated-dependencies:
- dependency-name: actions/stale
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-10-04 06:40:14 +00:00
Humble Chirammal
d1acae7209 deploy: update sidecars to latest versions available
Below sidecars are updated with this commit.

csi-provisioner: v3.3.0
csi-snapshotter: v6.1.0

This commit change the sidecar versions in build.env setup.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-09-29 10:24:26 +00:00
Niels de Vos
1eaf6af882 rebase: update minikube to 1.27.0
The fscrypt kernel module has been enabled, yay!

See-also: kubernetes/minikube#14783
Updates: #3310
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-09-23 16:32:41 +00:00
Humble Chirammal
1e0bd66108 rbd: make default fstype explicit to ext4
With the attacher sidecar update to v4.0.0 this has to be set
explictly.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-09-21 22:00:24 +00:00
Humble Chirammal
5fa606b14d deploy: update resizer to v1.6.0 version
This commit make deployment changes to consume v1.6.0
version of the resizer controller.

Reference:
https://github.com/kubernetes-csi/external-resizer/releases/tag/v1.6.0

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-09-20 10:32:28 +00:00
dependabot[bot]
4a1591236d rebase: bump github.com/onsi/ginkgo/v2 from 2.1.4 to 2.1.6
Bumps [github.com/onsi/ginkgo/v2](https://github.com/onsi/ginkgo) from 2.1.4 to 2.1.6.
- [Release notes](https://github.com/onsi/ginkgo/releases)
- [Changelog](https://github.com/onsi/ginkgo/blob/master/CHANGELOG.md)
- [Commits](https://github.com/onsi/ginkgo/compare/v2.1.4...v2.1.6)

---
updated-dependencies:
- dependency-name: github.com/onsi/ginkgo/v2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-09-15 23:32:43 +00:00
Yati Padia
36b061d426 rbd: get description from remote status
This commit gets the description from remote status
instead of local status.
Local status doesn't have ',' due to which we get
array index out of range panic.

Fixes: #3388

Signed-off-by: Yati Padia <ypadia@redhat.com>
Co-authored-by: shyam Ranganathan <srangana@redhat.com>
2022-09-14 12:06:01 +00:00
dependabot[bot]
e5c5646963 rebase: bump github.com/aws/aws-sdk-go-v2/service/sts
Bumps [github.com/aws/aws-sdk-go-v2/service/sts](https://github.com/aws/aws-sdk-go-v2) from 1.16.13 to 1.16.17.
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/v1.16.13...service/ram/v1.16.17)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/service/sts
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-09-14 08:56:18 +00:00
dependabot[bot]
9ddb4a784a rebase: bump github.com/IBM/keyprotect-go-client from 0.8.0 to 0.8.1
Bumps [github.com/IBM/keyprotect-go-client](https://github.com/IBM/keyprotect-go-client) from 0.8.0 to 0.8.1.
- [Release notes](https://github.com/IBM/keyprotect-go-client/releases)
- [Commits](https://github.com/IBM/keyprotect-go-client/compare/v0.8.0...v0.8.1)

---
updated-dependencies:
- dependency-name: github.com/IBM/keyprotect-go-client
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-09-14 07:23:57 +00:00
Humble Chirammal
fc89159daf doc: deprecate v3.5 as we are on v3.7
This commit mark v3.5 on deprecated release.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-09-14 04:26:03 +00:00
Humble Chirammal
a799075cfc doc: update doc for 3.7.1 release
updated doc for 3.7.1 release is available with this commit.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-09-14 04:26:03 +00:00
dependabot[bot]
d3a80f76d6 rebase: bump github.com/aws/aws-sdk-go from 1.44.82 to 1.44.96
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.82 to 1.44.96.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Changelog](https://github.com/aws/aws-sdk-go/blob/main/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.44.82...v1.44.96)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-09-13 18:45:16 +00:00
yati1998
b19705f260 rbd: implements getVolumeReplicationInfo
This commit implements getVolumeReplicationInfo
to get the last sync time and update it in volume
replication CR.

Signed-off-by: yati1998 <ypadia@redhat.com>
2022-09-13 14:17:10 +00:00
Rakshith R
a57859dfa4 rbd: use blocklist range cmd, fallback if it fails
This commit adds blocklist range cmd feature,
while fallbacks to old blocklist one ip at a
time if the cmd is invalid(not available).

Signed-off-by: Rakshith R <rar@redhat.com>
2022-09-13 13:10:32 +00:00
dependabot[bot]
3d04065d00 rebase: bump k8s.io/klog/v2 from 2.70.1 to 2.80.1
Bumps [k8s.io/klog/v2](https://github.com/kubernetes/klog) from 2.70.1 to 2.80.1.
- [Release notes](https://github.com/kubernetes/klog/releases)
- [Changelog](https://github.com/kubernetes/klog/blob/main/RELEASE.md)
- [Commits](https://github.com/kubernetes/klog/compare/v2.70.1...v2.80.1)

---
updated-dependencies:
- dependency-name: k8s.io/klog/v2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-09-13 07:56:49 +00:00
Prashanth Dintyala
2a6487cbf5 rbd: create token and use it for vault SA everytime possible
use TokenRequest API by default for vault SA even with K8s versions < 1.24

Signed-off-by: Prashanth Dintyala <vdintyala@nvidia.com>
2022-09-09 10:13:32 +00:00
Madhu Rajanna
76064d8e34 cephfs: retry subvolumegroup creation
Incase the  subvolumegroup is deleted
and recreated we need to restart the
cephcsi provisioner pod to clear cache
that cephcsi maintains. With this PR
if cephcsi sees NotFound error duing
subvolume creation it will reset the cache
for that filesystem so that in next RPC
call cephcsi will try to create the
subvolumegroup again

Ref: https://github.com/rook/rook/issues/10623

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-09-07 18:24:30 +00:00
Madhu Rajanna
e56621cd66 cephfs: fix subvolumegroup creation for multiple fs
In a cluster we can have multiple filesystem
for that we need to have a map of
subvolumegroups to check filesystem is created
nor not.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-09-07 18:24:30 +00:00
Madhu Rajanna
9d46478794 ci: use resync to sync helm charts
When a file on source is deleted same
need to be deleted on the destination,
with rsync we can achieve it.

fixes: #3329

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-09-06 11:22:51 +00:00
Madhu Rajanna
71dbc7dbb4 rbd: map only primary image
If the image is mirroring enabled
and primary consider it for mapping,
if the image is mirroring enabled but
not primary yet. return error message
until the image is marked as primary.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-09-06 10:40:12 +00:00
dependabot[bot]
9a54f05e66 rebase: bump google.golang.org/grpc from 1.48.0 to 1.49.0
Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.48.0 to 1.49.0.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.48.0...v1.49.0)

---
updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-09-05 16:32:37 +00:00
Humble Chirammal
586a9cc8ee rbd: change default FsGroupPolicy to "File" for RBD CSI driver
This commit change the default fsgroup policy for csi driver object
to "File" type which is the better/correct setting for the CSI volumes.
We have been using default value which is "ReadWriteOnceWithFSType".
with this change backward compatibility should be preserved.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-09-05 07:06:37 +00:00
Madhu Rajanna
038462ff43 cephfs: return success if metadata operation not supported
If the ceph cluster is of older version and doesnot
support metadata operation, Instead of failing
the request return the success if metadata
operation is not supported.

fixes #3347

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-08-29 18:37:53 +00:00
Rakshith R
40134772a7 rbd: modify stripSecret mechanism in logGRPC()
This commit updates csi-addons spec version
and modifies logging to strip replication
request secret using csi.StripSecret, then
with replication.protosanitizer if the former
fails. This is done in order to make sure
we strip csi and replication format of secrets.

Signed-off-by: Rakshith R <rar@redhat.com>
2022-08-29 11:18:15 +00:00
Madhu Rajanna
93f6923084 e2e: use ginkgo.timeout for e2e
ginkgo.timeout is introduced in
ginkgo timeout for overall test suite

https://onsi.github.io/ginkgo/
\MIGRATING_TO_V2#timeout-behavior

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-08-25 16:36:35 +00:00
Madhu Rajanna
ade8b91516 rebase: update gingko to v2 version
Ginkgo v1 is deprecated and was replaced
with the v2.

Ref: https://onsi.github.io/ginkgo/
MIGRATING_TO_V2#upgrading-to-ginkgo-20

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-08-25 16:36:35 +00:00
Madhu Rajanna
e3bf375035 rebase: update kubernetes to v1.25.0
update kubernetes to latest v1.25.0
release.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-08-25 16:36:35 +00:00
Rakshith R
f47839d73d rbd: improve kmip verifyResponse() error message
This commit uses %q instead %v in error messages
and adds result reason and message in kmip
verifyresponse().

Signed-off-by: Rakshith R <rar@redhat.com>
2022-08-24 07:58:57 +00:00
Rakshith R
eaa0e14cb2 rbd: fix bug in kmip kms Decrypt function
This commit fixes a bug in kmip kms Decrypt
function, where emd.DEK was fed in a Nonce
instead of emd.Nonce by mistake.

Signed-off-by: Rakshith R <rar@redhat.com>
2022-08-24 07:58:57 +00:00