Commit Graph

4232 Commits

Author SHA1 Message Date
Praveen M
8438381f1a e2e: test to verify the encrypted image size
This commit adds testcase for the create/resize/clone/restore operations
for a RBD block PVC and validate imageSize and deviceSize.

Signed-off-by: Praveen M <m.praveen@ibm.com>
2024-08-13 13:12:19 +00:00
Praveen M
a5af5a1382 rbd: add additional space for encrypted volumes
issue: when a block-mode pvc is created with encryption enabled
there is some space reserved for the encryption metadata.
Which doesn't allows users to write extact amount of data that
they have requested for.

solution: create pvc with extra space needed for the encryption
metadata.

The extra space is added during the CreateVolume and ExpandVolume
operations. And while returning the response remove the extra space
so the client/user gets the requested size reported.

Signed-off-by: Praveen M <m.praveen@ibm.com>
2024-08-13 13:12:19 +00:00
Nikhil-Ladha
71cbf3d7eb ci: add test for uncommitted changes in deploy directory
added test for uncommitted changes in deploy directory under go-test GH action.
Also, created a new make target named `make check-deploy-committed` that
can be used to verify the uncommitted changes.

Signed-off-by: Nikhil-Ladha <nikhilladha1999@gmail.com>
2024-08-13 12:17:43 +00:00
Praveen M
e150fd986b journal: fix empty csiCreationTimeKey value
This commit fixes the issue where the `csiCreationTimeKey`
field was missing during the rebuilding of the
`VolumeGroupJournalConfig` struct in the `Connect()` method,
which led to the `csi.creationtime` key not being stored in
the omap.

Signed-off-by: Praveen M <m.praveen@ibm.com>
2024-08-13 07:22:26 +00:00
Niels de Vos
869aaced7d rbd: convert rbdVolume to rbdSnapshot
After cloning the RBD snapshot, an rbdVolume is returned for the
CSI.Snapshot object. In order to use the rbdSnapshot.ToCSI() function,
the rbdVolume needs to be converted (back) to an rbdSnaphot.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-08-09 14:58:59 +00:00
Niels de Vos
503d10eb1a journal: store CreationTime for VolumeGroups
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-08-09 14:58:59 +00:00
Niels de Vos
6d1ab1b8d9 rbd: have GetCreationTime() return a time.Time struct
Do not use protobuf types when there is no need. Just use the standard
time.Time format instead.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-08-09 14:58:59 +00:00
Niels de Vos
2fd92573f4 ci: prevent incorrect pattern matching for skipping ci command
Mergify matches `body-raw` as regular expression, but that makes it
difficult (if not impossible) to match the `[` in a string.

Fixes: #4751
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-08-09 10:50:56 +00:00
Niels de Vos
d06497cf32 doc: add description of Mergify commands for merging a PR
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-08-09 10:06:28 +00:00
Niels de Vos
4a46ffd2b7 ci: detect [skip ci] in PR descriptions and prevent breakage
When Jenkins jobs are started by a comment, and if the PR contains
`[skip ci]` in the description, Jenkins does not run the requested job,
nor set a status for the job in the PR.

This causes Mergify to add the `ok-to-test` label again, instructing a
GitHub Action to add comments to start jobs in Jenkins. Once all
comments have been posted, the `ok-to-test` label is removed. Mergify
then notices that the jobs were not run, and adds the `ok-to-test` label
again.... Endlessly looping of adding the label, commenting and removing
the label as a result.

By reporting the brokenness of the PR description and marking the PR as
Draft, the looping is prevented.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-08-09 10:06:28 +00:00
dependabot[bot]
dabf3c4099 rebase: bump the github-dependencies group with 3 updates
Bumps the github-dependencies group with 3 updates: [github.com/IBM/keyprotect-go-client](https://github.com/IBM/keyprotect-go-client), [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) and [github.com/onsi/gomega](https://github.com/onsi/gomega).


Updates `github.com/IBM/keyprotect-go-client` from 0.14.3 to 0.15.1
- [Release notes](https://github.com/IBM/keyprotect-go-client/releases)
- [Changelog](https://github.com/IBM/keyprotect-go-client/blob/master/CHANGELOG.md)
- [Commits](https://github.com/IBM/keyprotect-go-client/compare/v0.14.3...v0.15.1)

Updates `github.com/aws/aws-sdk-go` from 1.55.4 to 1.55.5
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.55.4...v1.55.5)

Updates `github.com/onsi/gomega` from 1.34.0 to 1.34.1
- [Release notes](https://github.com/onsi/gomega/releases)
- [Changelog](https://github.com/onsi/gomega/blob/master/CHANGELOG.md)
- [Commits](https://github.com/onsi/gomega/compare/v1.34.0...v1.34.1)

---
updated-dependencies:
- dependency-name: github.com/IBM/keyprotect-go-client
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: github-dependencies
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: github-dependencies
- dependency-name: github.com/onsi/gomega
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: github-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-08-06 22:41:49 +00:00
dependabot[bot]
ee503fae54 rebase: bump golang.org/x/sys in the golang-dependencies group
Bumps the golang-dependencies group with 1 update: [golang.org/x/sys](https://github.com/golang/sys).


Updates `golang.org/x/sys` from 0.22.0 to 0.23.0
- [Commits](https://github.com/golang/sys/compare/v0.22.0...v0.23.0)

---
updated-dependencies:
- dependency-name: golang.org/x/sys
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: golang-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-08-06 14:37:43 +00:00
Niels de Vos
130e8b42f2 cephfs: use conf.InstanceID instead of global variable
RBD does not have a global CSIInstanceID variable anymore, there is no
need for CephFS to use one either.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-08-05 17:04:52 +00:00
Niels de Vos
dfb48bac17 util: add CSIDriver.GetInstanceID()
There has been some confusion about using different variables for the
InstanceID of the RBD-driver. By removing the global variable
CSIInstanceID, there should be no confusion anymore what variable to
use.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-08-05 17:04:52 +00:00
Niraj Yadav
0092a47586 doc: Remove podSecurityPolicy from helm docs
Fixes: #4714

Signed-off-by: Niraj Yadav <niryadav@redhat.com>
2024-08-05 09:53:42 +00:00
Niels de Vos
db6f54f75d ci: disable seccomp when running podman with GitHub Actions
Fedora 40 builds seems to require disabling seccomp. When enabled,
installing Golang inside the container fails.

See-also: containers/podman#21012
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-08-01 08:31:12 +00:00
Niels de Vos
a8d1b9dfe5 ci: use latest Fedora container-image
GitHub had issues with Fedora 40 when it was released. Hopefully this is
not the case anymore.

Closes: #4585
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-08-01 08:31:12 +00:00
Niels de Vos
e3697f4d3e doc: inform users that the OS in the container-image is updated
The Squid container-image that is used as base for the Ceph-CSI
container-image uses CentOS Stream 9.

Closes: #4659
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-07-31 14:12:59 +00:00
Madhu Rajanna
3a8981a735 rbd: add support to get volumegroupID
updated GetIDFromReplication to return
volumeGroupID if its present.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-07-31 12:16:42 +00:00
Madhu Rajanna
f7c78ae4fe rbd: update group Stringer method
updated the group stringer method
to have pool and namespace for
proper debugging/logging and to
use it with CLI as agrument as well.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-07-31 12:16:42 +00:00
Madhu Rajanna
37970ae212 rbd: add context to mirror interface
adding required ctx to the mirror
interface as ctx is required for
the volumegroup operations.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-07-31 12:16:42 +00:00
Madhu Rajanna
8788e5ec08 rbd: adapt to GetVolumeByID error message
GetVolumeByID already returning detailed
error message, the caller just need to return
it. No need to add duplicate details to error
message.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-07-31 12:16:42 +00:00
Madhu Rajanna
7e2e5ba2e5 rbd: flatten image in ModifyVolumeGroupMembership
in ModifyVolumeGroupMembership RPC call,
flatten the required images before adding it
to the group or else if the parent is not
mirror enabled adding a child to the group
will fail.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-07-31 12:16:42 +00:00
Madhu Rajanna
88a5c8a0eb vendor: update csiaddons spec
updating csiaddons spec to the
latest main.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-07-31 12:16:42 +00:00
Madhu Rajanna
e682f2cc73 rbd: add struct to error
updating HandleParentImageExistence function
to return more details error which includes
the pool/namespace/image name

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-07-31 12:16:42 +00:00
Madhu Rajanna
c773c98408 rbd: flatten image in CreateVolumeGroup
This commit adds support for flattenMode option
for volumegroup.
If the flattenMode is set to "force" in
volumegroupreplicationclass parameters,
cephcsi will add a task to flatten the image
if it has parent before adding it to the group.
This enable cephcsi to then mirror such images
after flattening them.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-07-31 12:16:42 +00:00
Madhu Rajanna
b222b773aa rbd: implement journalledObject for volumes
implement journalledObject interface to
return the journal objects of the volume.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-07-31 12:16:42 +00:00
Madhu Rajanna
a243cf52d4 rbd: return more descriptive error
updated GetVolumeByID to return more
descriptive error so that caller no
need to add more details in
the error message.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-07-31 12:16:42 +00:00
Praveen M
8fa3ac9fb3 cleanup: remove unnecessary error return type
Signed-off-by: Praveen M <m.praveen@ibm.com>
2024-07-31 06:56:32 +00:00
Praveen M
243a0fd0fb rbd: add volume locks for reclaimspace operations
This commit adds locks on reclaimspace operations to
prevent multiple process executing rbd sparsify/fstrim
on same volume.

Signed-off-by: Praveen M <m.praveen@ibm.com>
2024-07-31 06:56:32 +00:00
Madhu Rajanna
e6540989a5 doc: update pending release notes
updating pending release notes
to include ceph squid release.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-07-30 19:42:51 +00:00
Madhu Rajanna
b3ff07e4b5 build: update rook version and image
updating rook version and ceph image
for cluster to use squid.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-07-30 19:42:51 +00:00
Madhu Rajanna
c094699c32 build: update ceph to squid
updating ceph image and tag for
squid

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-07-30 19:42:51 +00:00
Niraj Yadav
439a03f21b cephfs: refactor locks using IOCtxLock interface
Signed-off-by: Niraj Yadav <niryadav@redhat.com>
2024-07-30 14:51:49 +00:00
Niraj Yadav
4445247690 rbd: use ioctx locks for key rotation
Signed-off-by: Niraj Yadav <niryadav@redhat.com>
2024-07-30 14:51:49 +00:00
Niraj Yadav
0bed833ef7 util: Add IOCtxLock interface
that abstracts rados Ioctx Lock/Unlocks

Signed-off-by: Niraj Yadav <niryadav@redhat.com>
2024-07-30 14:51:49 +00:00
dependabot[bot]
10fb3ef6cb rebase: bump the github-dependencies group with 3 updates
Bumps the github-dependencies group with 3 updates: [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go), [github.com/onsi/ginkgo/v2](https://github.com/onsi/ginkgo) and [github.com/onsi/gomega](https://github.com/onsi/gomega).


Updates `github.com/aws/aws-sdk-go` from 1.55.0 to 1.55.4
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.55.0...v1.55.4)

Updates `github.com/onsi/ginkgo/v2` from 2.19.0 to 2.19.1
- [Release notes](https://github.com/onsi/ginkgo/releases)
- [Changelog](https://github.com/onsi/ginkgo/blob/master/CHANGELOG.md)
- [Commits](https://github.com/onsi/ginkgo/compare/v2.19.0...v2.19.1)

Updates `github.com/onsi/gomega` from 1.33.1 to 1.34.0
- [Release notes](https://github.com/onsi/gomega/releases)
- [Changelog](https://github.com/onsi/gomega/blob/master/CHANGELOG.md)
- [Commits](https://github.com/onsi/gomega/compare/v1.33.1...v1.34.0)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: github-dependencies
- dependency-name: github.com/onsi/ginkgo/v2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: github-dependencies
- dependency-name: github.com/onsi/gomega
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: github-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-07-30 10:21:07 +00:00
Andreas
7afddb41d6 deploy: support omap data store in radosnamespace via cli argument
Signed-off-by: Andreas <zerotens@users.noreply.github.com>
2024-07-30 07:13:48 +00:00
Andreas
89ccbc8fba cephfs: support omap data store in radosnamespace via cli argument
Signed-off-by: Andreas <zerotens@users.noreply.github.com>
2024-07-30 07:13:48 +00:00
Madhu Rajanna
b185bfde4d rbd: refractor to use mirror interface
Refractoring code to use mirror
interface.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-07-26 12:14:20 +00:00
Madhu Rajanna
132f258569 rbd: add mirroring helper for image
Add helper methods for the rbd
image mirroring.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-07-26 12:14:20 +00:00
Madhu Rajanna
03de220380 rbd: add mirror interface
Adding mirror interface for rbd image
and group which can be used to manage
mirroring for both.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-07-26 12:14:20 +00:00
Rakshith R
14509c3e39 doc: add release note for flattenMode option
Signed-off-by: Rakshith R <rar@redhat.com>
2024-07-26 09:36:44 +00:00
Niels de Vos
aa88b4c4a0 rbd: implement CSI-Addons ControllerGetVolumeGroup operation
With the ControllerGetVolumeGroup operation the caller can verify that a
VolumeGroup exists, and validate the volumes that are part of it.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-07-26 08:12:49 +00:00
NymanRobin
da6279b59d cephfs: correct the id for cephfs locks
There was a discrepancy between the objectId
when creating the lock and when releasing the lock
this caused every lock to hang.

Signed-off-by: NymanRobin <robin.nyman@est.tech>
2024-07-25 18:12:22 +00:00
Praveen M
0a3ec18c30 rebase: update k8s.io packages to v0.30.3
Signed-off-by: Praveen M <m.praveen@ibm.com>
2024-07-25 15:28:04 +00:00
dependabot[bot]
e9c729b692 rebase: bump k8s.io/kubernetes in the k8s-dependencies group
Bumps the k8s-dependencies group with 1 update: [k8s.io/kubernetes](https://github.com/kubernetes/kubernetes).


Updates `k8s.io/kubernetes` from 1.30.2 to 1.30.3
- [Release notes](https://github.com/kubernetes/kubernetes/releases)
- [Commits](https://github.com/kubernetes/kubernetes/compare/v1.30.2...v1.30.3)

---
updated-dependencies:
- dependency-name: k8s.io/kubernetes
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: k8s-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-07-25 15:28:04 +00:00
Niels de Vos
74d434c3b7 rbd: check for valid UUID instead of name
It seems to be possible that the UUID was found, but the name is not
set. Checking on UUID makes the CreateVolumeGroup operation more
idempotent.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-07-25 12:18:24 +00:00
Niels de Vos
a3d457a8dd rbd: add VolumeGroup.ModifyVolumeGroupMembership CSI-Addons operation
The ModifyVolumeGroupMembership operation can be used to change the
volumes that are part of a VolumeGroup. Only empty VolumeGroups can be
removed, this operation is required to make that possible.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-07-25 11:46:04 +00:00
Niels de Vos
f9ab14e826 rbd: check if an image is part of a group before adding it
A RBD image can only be part of a single group. While an image is added
to a group, check if the image is already part of a group, and return an
error in case it is.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-07-24 15:55:07 +00:00