Commit Graph

4400 Commits

Author SHA1 Message Date
Nikhil-Ladha
98cf0780e1 cephfs: log clone progress
log cephfs clone progress report during cephfs clone
operation

Signed-off-by: Nikhil-Ladha <nikhilladha1999@gmail.com>
2024-11-22 08:04:50 +00:00
Madhu Rajanna
00d252e4ac rbd: use os.Remove to remove directory
using os.RemoveAll will remove everything
in the director after the Umount we should
be using os.Remove only to remove the empty
directory

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-11-21 10:18:56 +00:00
Madhu Rajanna
cd09266870 cephfs: use os.Remove to remove directory
using os.RemoveAll will remove everything
in the director after the Umount we should
be using os.Remove only to remove the empty
directory

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-11-21 10:18:56 +00:00
Madhu Rajanna
7cfeae579f cephfs: take lock on targetpath on node operation
We should not be dependent on the CO to ensure
that it will serialize the request instead of
that we need to have own internal locks to ensure
that we dont do concurrent operations for same
request.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-11-21 10:18:56 +00:00
Madhu Rajanna
b6bd8ca71a rbd: take lock on targetpath during node operation
We should not be dependent on the CO to ensure
that it will serialize the request instead of
that we need to have own internal locks to ensure
that we dont do concurrent operations for same
request.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-11-21 10:18:56 +00:00
dependabot[bot]
23c1d7c3e2 rebase: bump google.golang.org/protobuf from 1.35.1 to 1.35.2
Bumps google.golang.org/protobuf from 1.35.1 to 1.35.2.

---
updated-dependencies:
- dependency-name: google.golang.org/protobuf
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-11-20 12:02:22 +00:00
dependabot[bot]
86baab7322 rebase: bump github/codeql-action from 3.27.1 to 3.27.4
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.27.1 to 3.27.4.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](4f3212b617...ea9e4e3799)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-11-20 09:50:33 +00:00
dependabot[bot]
ecb62d6462 rebase: bump github.com/aws/aws-sdk-go-v2/service/sts
Bumps the github-dependencies group with 1 update: [github.com/aws/aws-sdk-go-v2/service/sts](https://github.com/aws/aws-sdk-go-v2).


Updates `github.com/aws/aws-sdk-go-v2/service/sts` from 1.32.4 to 1.33.1
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/service/s3/v1.33.1/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/v1.32.4...service/s3/v1.33.1)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/service/sts
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: github-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-11-20 07:54:09 +00:00
Rakshith R
d457840d21 rbd: set depthToAvoidFlatten to 3 during PVC-PVC clone
During PVC-PVC clone creation, parent of the datasource
image is flattened after checking for clone depth.
We need to account for data source image as well since
we're calculating depth from the parent image.
depthToAvoidFlatten = 3(datasource image + temp + final clone)

Signed-off-by: Rakshith R <rar@redhat.com>
2024-11-19 11:34:34 +00:00
Rakshith R
eea64fe1f9 rbd: remove checkFlatten() function
CephCSI should not flatten image that can be mounted
for use by the user.
`checkFlatten()` was called in a recovery code flow
of PVC restored from snapshot and was missed while
refractoring in https://github.com/ceph/ceph-csi/pull/2900

refer: #2900

Signed-off-by: Rakshith R <rar@redhat.com>
2024-11-19 11:34:34 +00:00
Antoine C
3e9b438e7c helm: add least privileges logic for secrets on ceph-csi-cephfs chart
this allows the encryption KMS config to be granted secret access with
a least privilges policy.

Signed-off-by: Antoine C <hi@acolombier.dev>
2024-11-18 15:28:23 +00:00
Antoine C
cc407d157e helm: support encryption config in ceph-csi-cephfs chart
this chart currently lack the ability to properly configure encryption,
as well as granting sufficent permission to allow controllers to access
secret when needed.

Signed-off-by: Antoine C <hi@acolombier.dev>
2024-11-18 15:28:23 +00:00
尤理衡 (Li-Heng Yu)
dc4ca2015e doc: fixed broken doc links
The deploy link in the README is broken.
Fixed more broken links requested by iPraveenParihar in #4958

Signed-off-by: 尤理衡 (Li-Heng Yu) <007seadog@gmail.com>
2024-11-18 09:36:41 +00:00
dependabot[bot]
d651011026 rebase: bump google.golang.org/grpc from 1.67.1 to 1.68.0
Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.67.1 to 1.68.0.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.67.1...v1.68.0)

---
updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Praveen M <m.praveen@ibm.com>
2024-11-14 07:18:51 +00:00
dependabot[bot]
925ea1970c rebase: bump the golang-dependencies group with 3 updates
Bumps the golang-dependencies group with 3 updates: [golang.org/x/crypto](https://github.com/golang/crypto), [golang.org/x/net](https://github.com/golang/net) and [golang.org/x/sys](https://github.com/golang/sys).


Updates `golang.org/x/crypto` from 0.28.0 to 0.29.0
- [Commits](https://github.com/golang/crypto/compare/v0.28.0...v0.29.0)

Updates `golang.org/x/net` from 0.30.0 to 0.31.0
- [Commits](https://github.com/golang/net/compare/v0.30.0...v0.31.0)

Updates `golang.org/x/sys` from 0.26.0 to 0.27.0
- [Commits](https://github.com/golang/sys/compare/v0.26.0...v0.27.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: golang-dependencies
- dependency-name: golang.org/x/net
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: golang-dependencies
- dependency-name: golang.org/x/sys
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: golang-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-11-12 17:10:45 +00:00
dependabot[bot]
710e7d273c rebase: bump the github-dependencies group with 2 updates
Bumps the github-dependencies group with 2 updates: [github.com/aws/aws-sdk-go-v2/service/sts](https://github.com/aws/aws-sdk-go-v2) and [github.com/Azure/azure-sdk-for-go/sdk/security/keyvault/azsecrets](https://github.com/Azure/azure-sdk-for-go).


Updates `github.com/aws/aws-sdk-go-v2/service/sts` from 1.32.3 to 1.32.4
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/v1.32.3...v1.32.4)

Updates `github.com/Azure/azure-sdk-for-go/sdk/security/keyvault/azsecrets` from 1.2.0 to 1.3.0
- [Release notes](https://github.com/Azure/azure-sdk-for-go/releases)
- [Changelog](https://github.com/Azure/azure-sdk-for-go/blob/main/documentation/release.md)
- [Commits](https://github.com/Azure/azure-sdk-for-go/compare/v1.2...sdk/azcore/v1.3.0)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/service/sts
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: github-dependencies
- dependency-name: github.com/Azure/azure-sdk-for-go/sdk/security/keyvault/azsecrets
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: github-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-11-12 10:38:23 +00:00
dependabot[bot]
89ab2e78e8 rebase: bump golang.org/x/oauth2 in /actions/retest
Bumps [golang.org/x/oauth2](https://github.com/golang/oauth2) from 0.23.0 to 0.24.0.
- [Commits](https://github.com/golang/oauth2/compare/v0.23.0...v0.24.0)

---
updated-dependencies:
- dependency-name: golang.org/x/oauth2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-11-12 10:06:07 +00:00
dependabot[bot]
1981d7fb38 rebase: bump github/codeql-action from 3.27.0 to 3.27.1
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.27.0 to 3.27.1.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](662472033e...4f3212b617)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-11-12 09:35:15 +00:00
Niels de Vos
d98516e9d8 rbd: add locking for VolumeGroupSnapshot operations
Add VolumeGroupLocks in the CSI Controller Server so that operations are
protected against concurrent requests for the same VolumeGroupSnapshot.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-11-12 09:28:30 +00:00
Niels de Vos
f3d40f9e5a rbd: cleanup inconsistent state in reserveSnap() after a failure
`reserveSnap()` can potentially fail halfway through, in that case it
needs to undo the snapshot reservation and restore modified attributes
of the snapshot.

Fixes: #4945
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-11-11 13:39:05 +00:00
Niels de Vos
cea8bf8110 rbd: set SnapshotGroupID on each Snapshot of a VolumeGroupSnapshot
Without the SnapshotGroupID in the Snapshot object, Kubernetes CSI does
not know that the Snapshot belongs to a group. In that case, it allows
the deletion of the Snapshot, which should be denied.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-11-06 11:37:44 +00:00
Niels de Vos
ec1e7a4ee0 rbd: expose the GroupControllerService
When the GroupSnapGetInfo go-ceph function is supported by librbd, the
Group Controller Servive and VolumeGroupSnapshot capabilities can be
exposed to the Container Orchestrator.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-11-06 11:37:44 +00:00
Niels de Vos
e34dceff27 rbd: implement CSI Group Controller Server
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-11-06 11:37:44 +00:00
Niels de Vos
e011e74b9d rbd: fix snapshot deletion by resolving image names correctly
When creating a Snapshot with the new NewSnapshotByID() function, the
name of the RBD-image that is created is the same as the name of the
Snapshot. The `RbdImageName` points to the name of parent image, which
causes deleting the Snapshot to delete the parent image instead.

Correcting the `RbdImageName` and setting it to the `RbdSnapName` makes
sure that upon deletion, the Snapshot RBD-image is removed, and not the
parent image.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-11-06 11:37:44 +00:00
Niels de Vos
fdccba1f33 rbd: add Manager.GetVolumeGroupSnapshotByName
The Group Controller Server may need to fetch a VolumeGroupSnapshot that
was statically provisioned. In that case, only the name of the
VolumeGroupSnapshot is known and should be resolved to an object.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-11-06 11:37:44 +00:00
Niels de Vos
ad381c4ff0 rbd: implement Manager.GetVolumeGroupSnapshotByID
The GetVolumeGroupSnapshotByID function makes it possible to get a
VolumeGroupSnapshot object from the Manager by passing a request-id.
This makes it simple for the Group Controller Server to check if a
VolumeGroupSnapshot already exists, so it is not needed to try and
re-create an existing one.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-11-06 11:37:44 +00:00
Niels de Vos
7563f4285d rbd: add manager.CreateVolumeGroupSnapshot()
Implement the CreateVolumeGroupSnapshot for the rbd.Manager. A Group
Controller Server can use the rbd.Manager to create VolumeGroupSnapshots
in an easy an idempotent way.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-11-06 11:37:44 +00:00
Niels de Vos
9bea3feff1 rbd: add manager GetSnapshotByID and SnapshotResolver interface
A (CSI) VolumeGroupSnapshot object contains references to Snapshot IDs
(or CSI Snapshot handles). In order to work with a VolumeGroupSnapshot
struct, the Snapshot IDs need to be resolved into rbdSnapshot structs.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-11-06 11:37:44 +00:00
Niels de Vos
455a90e9f4 rbd: add VolumeGroupSnapshot type
The VolumeGroupSnapshot type will be used by the rbd.Manager to create,
inspect and delete VolumeGroupSnapshos.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-11-06 11:37:44 +00:00
Niels de Vos
efb7bccaea rbd: add VolumeGroup.CreateSnapshots() implementation
When the rbd.Manager creates a VolumeGroupSnapshot, each RBD-snapshot
that is created as part of the RBD-group needs to be cloned into its own
RBD-image that will be used as a CSI Snapshot.

The VolumeGroup.CreateSnapshots() creates the RBD-group snapshot and
returns a list of the Snapshot structs.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-11-06 11:37:44 +00:00
Niels de Vos
20fadf2016 rbd: add rbdVolume.NewSnapshotByID to clone images by RBD snapshot-id
The NewSnapshotByID() function makes it possible to clone a new Snapshot
from an existing RBD-image and the ID of an RBD-snapshot on that image.

This will be used by the VolumeGroupSnapshot feature, where the ID of an
RBD-snapshot is obtained for the RBD-snapshot on the RBD-images.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-11-06 11:37:44 +00:00
Niels de Vos
9808408340 rbd: pass CSI-drivername to volume group instead of journal instance
Each object is responsible for maintaining a connection to the journal.

By sharing a single journal, cleanup of objects becomes more complex as
the journal is used in deferred functions and only the last should
destroy the journal connection resources.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-11-06 11:37:44 +00:00
Niels de Vos
29bf5797b0 rbd: add .requestName to the commonVolumeGroup struct
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-11-06 11:37:44 +00:00
Niels de Vos
4b13e9132b rbd: have GetVolumeGroup() return an empty volume group if it was not found
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-11-06 11:37:44 +00:00
Niels de Vos
5d5171c7d7 journal: store csi.groupid for snapshots
Commit 95733b3a9 introduced the `StoreGroupID()` function, but that
unfortunately set an empty key in the journal.

Passing the `csiGroupIDKey` key (with value `csi.groupid`) caused
setting `csi.csi.groupid` as a key. Reading the value back with the
right `csi.groupid` key always returned an empty value.

Fixes: 95733b3a9 "journal: add option to store the groupID"
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-11-06 11:37:44 +00:00
Niels de Vos
6d88e0a4c7 rbd: close the RBD-image after adding it to a VolumeGroup
When the image is not closed, it keeps a watch open. This prevents the
CSI Controller to delete the Volume, as there is still a user of it.

Fixes: f9ab14e826 "rbd: check if an image is part of a group before adding it"
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-11-06 11:37:44 +00:00
Madhu Rajanna
b4592a55eb rbd: parse IP address
The address we get from ceph
contains the ip in the format
of 10.244.0.1:0/2686266785 we
need to extract the client IP
from this address, we already
have a helper to extract it,
This makes the helper more generic
can be reused by multiple packages
in the fence controller.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-11-06 09:48:45 +00:00
Madhu Rajanna
facf805941 rbd: implement GetFenceClients
implemented GetFenceClients which
connects to the ceph cluster and
returns the ceph clusterID and the
clientaddress that is used for rados
connection.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-11-06 09:48:45 +00:00
Madhu Rajanna
ba8c5a359c util: add GetAddrs method
added GetAddrs to get the client
Adress of the rados connection
which is helpful for NetworkFencing

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-11-06 09:48:45 +00:00
Madhu Rajanna
1246e2fac7 rebase: update go-ceph to latest commit
updating go-ceph to the latest commit to
pull the new function to get client address

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-11-06 09:48:45 +00:00
Madhu Rajanna
fdc74973d8 rbd: register GET_CLIENTS_TO_FENCE caps
register Capability_NetworkFence_
GET_CLIENTS_TO_FENCE capability and
start a NetworkFence controllers
as part of rbd nodeplugin.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-11-06 09:48:45 +00:00
Madhu Rajanna
34dfd0adb8 rebase: update csi-addons spec
updating csi-addons spec to the
latest main which GetFenceClients
API.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-11-06 09:48:45 +00:00
dependabot[bot]
7b06b0f218 rebase: bump the github-dependencies group with 2 updates
Bumps the github-dependencies group with 2 updates: [github.com/onsi/ginkgo/v2](https://github.com/onsi/ginkgo) and [github.com/onsi/gomega](https://github.com/onsi/gomega).


Updates `github.com/onsi/ginkgo/v2` from 2.20.2 to 2.21.0
- [Release notes](https://github.com/onsi/ginkgo/releases)
- [Changelog](https://github.com/onsi/ginkgo/blob/master/CHANGELOG.md)
- [Commits](https://github.com/onsi/ginkgo/compare/v2.20.2...v2.21.0)

Updates `github.com/onsi/gomega` from 1.34.2 to 1.35.1
- [Release notes](https://github.com/onsi/gomega/releases)
- [Changelog](https://github.com/onsi/gomega/blob/master/CHANGELOG.md)
- [Commits](https://github.com/onsi/gomega/compare/v1.34.2...v1.35.1)

---
updated-dependencies:
- dependency-name: github.com/onsi/ginkgo/v2
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: github-dependencies
- dependency-name: github.com/onsi/gomega
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: github-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-11-05 13:06:23 +00:00
Niraj Yadav
1c02e69ba4 rbd: Add timeout for cryptsetup commands
This PR modifies the execCryptSetupCommand so that
the process is killed in an event of lock timeout.

Useful in cases where the volume lock is released but
the command is still running.

Signed-off-by: Niraj Yadav <niryadav@redhat.com>
2024-11-05 11:39:59 +00:00
Niels de Vos
c451997762 e2e: detect support for VolumeGroupSnapshot in librbd
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-11-04 15:38:18 +00:00
Niels de Vos
b59a701777 e2e: add tests for RBD VolumeGroupSnapshots
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-11-04 15:38:18 +00:00
Madhu Rajanna
f4f03044f2 e2e: enhance volumegroupsnapshot test
update the e2e for volumegroupsnapshot
to create application pods from clone pvc
and delete the pods once we are able to
create the pods.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-11-04 10:28:27 +00:00
dependabot[bot]
a7959d4721 rebase: bump k8s.io/api in /api in the k8s-dependencies group
Bumps the k8s-dependencies group in /api with 1 update: [k8s.io/api](https://github.com/kubernetes/api).

Updates `k8s.io/api` from 0.31.1 to 0.31.2
- [Commits](https://github.com/kubernetes/api/compare/v0.31.1...v0.31.2)

---
updated-dependencies:
- dependency-name: k8s.io/api
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: k8s-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Praveen M <m.praveen@ibm.com>
2024-10-31 10:59:08 +00:00
Madhu Rajanna
00c7fb3136 e2e: log stderror in execCommandInPodAndAllowFail
If the command we try to execute fails, log
the stdError along with error for better debugging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-10-29 17:30:49 +00:00
Madhu Rajanna
98471b27fc e2e: add a test case of ROX PVC-PVC clone
adding a test case to test below case for RBD

* Create PVC and application
* Scale down application pod
* Create PVC-PVC ROX clone
* Try to create many pods that use the ROX PVC
* Try to write the data and verify only read
access.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-10-29 17:30:49 +00:00