Commit Graph

4142 Commits

Author SHA1 Message Date
Madhu Rajanna
307a2f130b ci: add volumesnapshotclasses for externalstorage
added volumesnapshotclasses yaml and script for
external-storage test

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-04-04 12:55:03 +00:00
Madhu Rajanna
66f71aea3a ci: remove auto merge condition
auto merge for release-3.11 is already
convered with regex check and we cannot
auto merge PR to the devel branch.
Removing unwanted checks.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-04-04 08:08:30 +02:00
Madhu Rajanna
61eec7e9c8 ci: remove condition for dependabot
dependabot will not create PR on the
release branch and for that reason
removing unused rule

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-04-04 08:08:30 +02:00
Madhu Rajanna
884d49e2ab ci: remove mergify rules for 3.11
we dont need a new mergify rule for
3.11 branch and the default mergirfy
with regex for `^(release-.+)$` branch
will cover it.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-04-04 08:08:30 +02:00
karthik-us
01ca536a56 ci: upgrade_version update
Updating the CSI_UPGRADE_VERSION to the latest release.

Signed-off-by: karthik-us <ksubrahm@redhat.com>
2024-04-03 12:00:46 +00:00
Madhu Rajanna
5718053e9c ci: use golang 1.21 for retest action
use golang 1.21 for resest action
dockerfile.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-04-02 14:10:11 +00:00
Madhu Rajanna
1e0254daba util: use slices package for contains
use slices package to check whether
the slice contains the element or not.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-04-02 14:10:11 +00:00
Niels de Vos
86b5db90bc cleanup: rename EncryptionTypeString() to EncryptionType.String()
This makes it easier to log the EncryptionType as string, or int,
whatever is preferred. Standard fmt formatting notations like %s or %d
can be used now.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-04-02 14:04:10 +00:00
Niels de Vos
5a6556c4d4 cleanup: destroy connections after .Copy() an other one
Everytime a connection is copied with the .Copy() function, it needs to
be destroyed once the object is not needed anymore. This was not done
consistently, a few more locations require the freeing of the connection
resources.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-04-02 11:11:20 +00:00
Rakshith R
4aa53c823e ci: modify mergifyio rules for new release-v3.11
This commit adds support for v3.11 backports
and removes support for v3.9 backports.

Signed-off-by: Rakshith R <rar@redhat.com>
2024-04-02 08:07:30 +00:00
Rakshith R
97bc20ae5a ci: update pr-commentor rules matrix
This commit adds rules for release-v3.11
and removes rules for release-v3.9.

Signed-off-by: Rakshith R <rar@redhat.com>
2024-04-02 08:07:30 +00:00
Rakshith R
04670a8b4d doc: add steps for upgrading from 3.10 to 3.11
This commit adds steps for upgrading from 3.10
to 3.11.

Signed-off-by: Rakshith R <rar@redhat.com>
2024-04-02 08:07:30 +00:00
Rakshith R
0e4991ac8b doc: update README for v3.11.0 release
This commit updates README to deprecate 3.9.x release,
add 3.10.1/2 release versions and pin latest release
to v3.11.0

Signed-off-by: Rakshith R <rar@redhat.com>
2024-04-02 08:07:30 +00:00
Praveen M
3c8ea475ec doc: csi driver object options
Signed-off-by: Praveen M <m.praveen@ibm.com>
2024-04-01 09:27:01 +00:00
Niels de Vos
8e57efad98 cleanup: use standard Golang "slices" instead of k8s package
The "slices" package has been introduced in Go 1.21 and can be used
instead of the Kubernetes package that will be replaced by the standard
package at one point too.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-03-29 12:42:24 +00:00
Praveen M
33a888f9ec helm: fix seLinuxMount option for csi driver
This commit fixes the typo from `.Values.seLinuxMount` to
`.Values.CSIDriver.seLinuxMount` used in helm charts.

Signed-off-by: Praveen M <m.praveen@ibm.com>
2024-03-29 10:46:18 +00:00
Niels de Vos
3df396e6f1 rbd: add extra logging while cleaning up snapshots
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-03-28 11:54:28 +00:00
Niels de Vos
ba05c0f5f1 cleanup: reformat generateVolFromSnap() to rbdSnapshot.toVolume()
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-03-28 11:54:28 +00:00
Niels de Vos
a517290ea7 rbd: let parseVolCreateRequest() return a connected rbdVolume
By returning a connected rbdVolume in parseVolCreateRequest(), the
CreateVolume() function can be simplified a little. There is no need to
call the additional Connect() and detect failures with it.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-03-28 11:54:28 +00:00
Niels de Vos
7b2b125b18 rbd: free snapshot resources after allocation
Not all snapshot objects are free'd correctly after they were allocated.
It is possible that some connections to the Ceph cluster were never
closed. This does not need to be a noticeable problem, as connections
are re-used where possible, but it isn't clean either.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-03-28 11:54:28 +00:00
Niels de Vos
18162c71bc cleanup: do not pass an empty snapshot to genSnapFromSnapID()
Just like GenVolFromVolID() the genSnapFromSnapID() function can return
a snapshot. There is no need to allocated an empty snapshot and pass
that to the genSnapFromSnapID() function.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-03-28 11:54:28 +00:00
parth-gr
063319f6e5 rbd: make pool optional in rbd sc if topologyconstraints are present
if rbd storage class is created with topologyconstraintspools
replicated pool was still mandatory, making the pool optional if the
topologyconstraintspools is requested

Closes: https://github.com/ceph/ceph-csi/issues/4380

Signed-off-by: parth-gr <partharora1010@gmail.com>
2024-03-22 13:15:50 +00:00
NymanRobin
5224d58c13 cephfs: add support for encryption in ceph-csi-cephfs chart
the chart currently lacks access to configmap and secrets
this causes the mounting of encrypted file systems to fail

Signed-off-by: NymanRobin <nyman.robin@gmail.com>
2024-03-21 14:58:33 +00:00
Ruslan Khizhnyak
d56c9abbce helm: CSIDriver add labels and seLinuxMount disabling method
Signed-off-by: Ruslan Khizhnyak <rkhizhnyak@ptsecurity.com>
2024-03-21 10:07:23 +00:00
Madhu Rajanna
cd18490451 rebase: update go-ceph to latest commit
updating go-ceph to latest commit to
pull the changes required for ceph fs
quiesce. This is also updating aws
sdk dependency.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-03-21 06:16:42 +00:00
Madhu Rajanna
aa2094ba30 cephfs: unit test for validateVolumeGroupSnapshotRequest
Added unit test for
validateVolumeGroupSnapshotRequest API which
validates the input VolumeGroupSnapshotRequest
request

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-03-21 06:16:42 +00:00
Madhu Rajanna
728a7f5ac7 util: add UnimplementedGroupControllerServer
adding UnimplementedGroupControllerServer to
the DefaultControllerServer struct to avoid
build errors when some non mandatory RPC's
are not implemented.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-03-21 06:16:42 +00:00
Madhu Rajanna
0f724480f5 cephfs: implement DeleteVolumeGroupSnapshot RPC
implemented DeleteVolumeGroupSnapshot RPC which
does below operations

* Basic request validation
* Get the snapshotId's and volumeId's
mapping reserved for the UUID
* Delete snapshot and remove its mapping
from the omap
* Repeat above steps until all the mapping
are removed
* Remove the reserved uuid from the omap
* Reset the filesystem quiesce, This might be
required as cephfs doesnt provide any options to
remove the quiesce, if we get any request with same
ID again we can reuse the quiesce API for same set-id
* Return success if the received error is
Pool not found or key not found.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-03-21 06:16:42 +00:00
Madhu Rajanna
df770e4139 cephfs: implement CreateVolumeGroupSnapshot RPC
implemented CreateVolumeGroupSnapshot RPC which
does below operations

* Basic request validation
* Reserve the UUID for the group name
* Quiesce the filesystem for all the subvolumes
from the input volumeId's
* Take the snapshot for all the input volumeId's
* Add the mapping between volumeId's and snapshot
Id's in omap
* Release the quiesce for the filesystem for
all the subvolumes from the input volumeId's

Undo all the operations if anything fails.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-03-21 06:16:42 +00:00
Madhu Rajanna
ffb2b1144d cephfs: add helper for group options
volumegroup.go holders all the helpers
to extra the group details from the request
and also to extra group details from the
groupID.

This also provide helpers to reserve group
for the request Name and also an undo function
incase if somethings goes wrong and we need to
cleanup the reserved omap entries.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-03-21 06:16:42 +00:00
Madhu Rajanna
86bf74bb5c cephfs: add helper function to getVolumeOptions
added helper function to extract basic
details from the parameters related to volume
options.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-03-21 06:16:42 +00:00
Madhu Rajanna
6a4c45deeb cephfs: add helper for quiesce api
added helper function which calls
the go-ceph API for the quiesce
operations.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-03-21 06:16:42 +00:00
Madhu Rajanna
eff0fe3a23 cephfs: add error for quiesce operation
added ErrInProgress to indicate the
the quiesce operation is in progress.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-03-21 06:16:42 +00:00
Madhu Rajanna
ef25a816a7 cephfs: add locks for volumegroup
Adding a lock for the volumegroup so
that we can take care of serializing
the same requests to ensure same requests
are not served in parallel.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-03-21 06:16:42 +00:00
Madhu Rajanna
b30da094b0 build: add ceph_preview to GO_TAGS_LIST
added required ceph_preview tag to the
GO_TAGS_LIST in Makefile which is
required for FSQuiesce API.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-03-21 06:16:42 +00:00
Madhu Rajanna
6ec86879e6 cephfs: register group controller
register the group controller service
for the cephfs.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-03-21 06:16:42 +00:00
Madhu Rajanna
ff6eda0de1 cephfs: initialize VolumeGroupJournal
initialize VolumeGroupJournal which is
required for volumegroup rados communication

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-03-21 06:16:42 +00:00
Madhu Rajanna
445de7926d cephfs: add validateCreateVolumeGroupSnapshotRequest
added validateCreateVolumeGroupSnapshotRequest
to validate the CreateVolumeGroupSnapshotRequest
request and ensure that all the requirement
options are set. if not, reject the RPC request.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-03-21 06:16:42 +00:00
Madhu Rajanna
68e93a31cc journal: fix connection problem with groupjournal
Same group jounral config need to be reused
for multiple connection where different monitors
and users are used, for that reason create a unique
connection each time.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-03-21 06:16:42 +00:00
Madhu Rajanna
f17ea38736 cephfs: advertise group snapshot capability
Advertise VOLUME_GROUP_SNAPSHOT capability
from the cephfs driver.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-03-21 06:16:42 +00:00
Madhu Rajanna
6b3484f285 cephfs: add volumegroup service capability
Add GROUP_CONTROLLER_SERVICE capabilities to
the GetPluginCapabilities of the cephFS
plugin.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-03-21 06:16:42 +00:00
dependabot[bot]
3ad922dbae rebase: bump the k8s-dependencies group in /api with 1 update
Bumps the k8s-dependencies group in /api with 1 update: [k8s.io/api](https://github.com/kubernetes/api).


Updates `k8s.io/api` from 0.29.2 to 0.29.3
- [Commits](https://github.com/kubernetes/api/compare/v0.29.2...v0.29.3)

---
updated-dependencies:
- dependency-name: k8s.io/api
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: k8s-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-03-19 12:31:13 +00:00
dependabot[bot]
5b9730ce6e rebase: bump the k8s-dependencies group with 1 update
Bumps the k8s-dependencies group with 1 update: [k8s.io/kubernetes](https://github.com/kubernetes/kubernetes).

Updates `k8s.io/kubernetes` from 1.29.2 to 1.29.3
- [Release notes](https://github.com/kubernetes/kubernetes/releases)
- [Commits](https://github.com/kubernetes/kubernetes/compare/v1.29.2...v1.29.3)

---
updated-dependencies:
- dependency-name: k8s.io/kubernetes
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: k8s-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-03-19 11:43:39 +00:00
Praveen M
b9543d3fd3 helm: update template for rbd volumegroupsnapshot
This commit updates template for rbd VolumeGroupSnapshot.
The value is set to false by default.

Signed-off-by: Praveen M <m.praveen@ibm.com>
2024-03-18 17:00:45 +00:00
Praveen M
bd07dd4e71 rbd: deployment changes to support VGS
Signed-off-by: Praveen M <m.praveen@ibm.com>
2024-03-18 17:00:45 +00:00
Niels de Vos
991343d9e5 cleanup: do not pass EncodingVersion to GenerateVolID()
The only encoding version that exists is `1`. There is no need to have
multiple constants for that version across different packages. Because
there is only one version, `GenerateVolID()` does not really require it,
and it can use a default version.

If there is a need in the future to support an other encoding version,
this can be revisited with a cleaner solution.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-03-18 06:23:28 +00:00
Niels de Vos
c32dfc0ae6 cleanup: correct typo in NewCSIVolumeroupJournal() function
The name of the function should be `NewCSIVolumeGroupJournal()`.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-03-15 11:05:10 +00:00
muxuelan
4f04748c87 rbd: support nbd on euler or arm
Signed-off-by: muxuelan <muxuelan@cmss.chinamobile.com>
2024-03-15 10:39:50 +00:00
Niels de Vos
6f0d9a5d59 deploy: include ServiceAccount in the NFS provisioner RBAC artifact
It seems that the ServiceAccount was not created anymore, this causes
problems with provisioning volumes.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-03-15 08:02:25 +00:00
Niels de Vos
a469a66f9d e2e: replace quoted namespace in templates too
Some templates are now generated with the API, and these include
namespaces as "quotes" values. Namespace replacing in the templates need
to replace both the unquoted and quoted strings.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-03-15 08:02:25 +00:00