Commit Graph

4053 Commits

Author SHA1 Message Date
Nikhil-Ladha
7a46d253a2 ci: add test for uncommitted changes in deploy directory
added test for uncommitted changes in deploy directory under go-test GH action.
Also, created a new make target named `make check-deploy-committed` that
can be used to verify the uncommitted changes.

Signed-off-by: Nikhil-Ladha <nikhilladha1999@gmail.com>
(cherry picked from commit 71cbf3d7eb)
2024-08-14 06:38:43 +00:00
Praveen M
41fe80d303 ci: update centos stream 8 baseurl
Since CentOS Stream 8 is EOL, this commit updates the
config to use vault.centos.org for CentOS Stream 8.
This should be removed once the base image (ceph) is
updated to a version with a newer CentOS.

Signed-off-by: Praveen M <m.praveen@ibm.com>
(cherry picked from commit 5809628523)
2024-07-16 15:29:24 +00:00
Praveen M
6753bbfba3 util: exclude empty label values for crushlocation map
This commit resolves a bug where node labels with empty values
are processed for the crush_location mount option,
leading to invalid mount options and subsequent mount failures.

Signed-off-by: Praveen M <m.praveen@ibm.com>
(cherry picked from commit f11fa815c8)
2024-07-16 15:29:24 +00:00
guangwu
47d17c9443 cephfs: close dir
Signed-off-by: guoguangwu <guoguangwug@gmail.com>
(cherry picked from commit 9574fa881b)
2024-05-28 12:02:45 +00:00
Madhu Rajanna
9d83b50c3e ci: install ruby-devel in test container
The tests are failing due to missing
ruby-devel package in the base image

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit 90c6be089b)
2024-05-17 17:21:03 +00:00
Niels de Vos
4c7d419336 ci: require k8s-1.30 CI jobs for release-v3.11 and devel branches
Signed-off-by: Niels de Vos <ndevos@ibm.com>
(cherry picked from commit 53579991eb)
2024-05-16 08:57:07 +00:00
Niels de Vos
ac7c117323 ci: do not require k8s-1.26 jobs for release-v3.11
Signed-off-by: Niels de Vos <ndevos@ibm.com>
(cherry picked from commit 3d3c0aae6c)
2024-05-16 08:57:07 +00:00
Niels de Vos
cee13bf273 ci: trigger k8s-1.30 jobs for current Ceph-CSI versions
Run CI jobs with Kubernetes 1.30 for the devel branch and upcoming
releases.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
(cherry picked from commit c558588bd8)
2024-05-16 08:57:07 +00:00
Niels de Vos
67ac2a58ff ci: do not run k8s-1.26 tests for release-v3.11 and devel
The k8s-external-storage/1.26 CI job does not work with the versions
that are part of the release-v3.11 and devel branches.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
(cherry picked from commit 6009c28c30)
2024-05-16 08:57:07 +00:00
NymanRobin
ca713945ad cephfs: upgrade fscrypt version to fix concurrency issue
In older versions of fscrypt there is a race condition
when multiple encrypted cephfs instances are deployed
simultaneously.

Signed-off-by: NymanRobin <robin.nyman@est.tech>
(cherry picked from commit 3073409695)
2024-05-15 07:59:38 +00:00
Praveen M
3805bea505 deploy: update CSI sidecars to latest versions available
Below sidecars are updated with latest available versions

csi-node-driver-registrar: v2.10.1
csi-resizer: v1.10.1
csi-provisioner: v4.0.1
csi-attacher: v4.5.1
csi-snapshotter: v7.0.2

Signed-off-by: Praveen M <m.praveen@ibm.com>
(cherry picked from commit b095e0441a)
2024-04-26 07:25:52 +00:00
Niels de Vos
7c557d2610 build: use Go 1.21.9 to include latest security fixes
Update from v1.21.5 to latest v1.21.9, to include the following fixes:

go1.21.6 (released 2024-01-09) includes fixes to the compiler, the
runtime, and the crypto/tls, maps, and runtime/pprof packages.

go1.21.7 (released 2024-02-06) includes fixes to the compiler, the go
command, the runtime, and the crypto/x509 package.

go1.21.8 (released 2024-03-05) includes security fixes to the
crypto/x509, html/template, net/http, net/http/cookiejar, and net/mail
packages, as well as bug fixes to the go command and the runtime.

go1.21.9 (released 2024-04-03) includes a security fix to the net/http
package, as well as bug fixes to the linker, and the go/types and
net/http packages.

See-also: https://go.dev/doc/devel/release#go1.21.0
Signed-off-by: Niels de Vos <ndevos@ibm.com>
(cherry picked from commit ae7f513816)
2024-04-25 19:52:51 +00:00
Niels de Vos
dc33450a84 build: use Fedora 39 as base image for test container
It seems GitHub has an issue with the Fedora 40 container image,
extracting the Golang tarball fails. The Fedora 39 image does not have
this problem, so use that for the time being.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
(cherry picked from commit 60e2527917)
2024-04-25 14:00:32 +00:00
Rakshith R
2f19479981 e2e: validate PVC-PVC clone creation with deleted parent snap
This commit modifies a test case to check creation of
PVC-PVC clone of a restored PVC when parent snapshot
is deleted.

Signed-off-by: Rakshith R <rar@redhat.com>
(cherry picked from commit 1bb78fdf43)
2024-04-25 07:21:43 +00:00
Rakshith R
3c68748c11 rbd: add ParentInTrash parameter in rbdImage struct
This commit adds ParentInTrash parameter in rbdImage struct
and makes use of it in getParent() function in order to avoid
error in case the parent is present but in trash.

Signed-off-by: Rakshith R <rar@redhat.com>
(cherry picked from commit c34b31ee05)
2024-04-25 07:21:43 +00:00
Praveen M
2ccb57a1cc cleanup: client cert decoding is not required
Signed-off-by: Praveen M <m.praveen@ibm.com>
(cherry picked from commit 29f1fbeb84)
2024-04-15 13:53:36 +00:00
Madhu Rajanna
a3a10855da ci: fix secret name in rbd storageclass
use rbd secret for rbd storageclass

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit 9fb3743280)
2024-04-04 13:21:52 +00:00
Madhu Rajanna
efd2414cce ci: add volumesnapshotclasses for externalstorage
added volumesnapshotclasses yaml and script for
external-storage test

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit 307a2f130b)
2024-04-04 13:21:52 +00:00
Rakshith R
2ee72f0391 revert: revert to 3.11-canary
This commit makes template changes for
3.11 release branch to use 3.11-canary tag.

Signed-off-by: Rakshith R <rar@redhat.com>
2024-04-04 07:51:21 +00:00
Rakshith R
bc24b5eca8 deploy: update templates to v3.11.0
This commit updates templates to new
v3.11.0 release.
Pending Release Notes is also reset.

Signed-off-by: Rakshith R <rar@redhat.com>
2024-04-02 13:03:08 +00:00
Rakshith R
4aa53c823e ci: modify mergifyio rules for new release-v3.11
This commit adds support for v3.11 backports
and removes support for v3.9 backports.

Signed-off-by: Rakshith R <rar@redhat.com>
2024-04-02 08:07:30 +00:00
Rakshith R
97bc20ae5a ci: update pr-commentor rules matrix
This commit adds rules for release-v3.11
and removes rules for release-v3.9.

Signed-off-by: Rakshith R <rar@redhat.com>
2024-04-02 08:07:30 +00:00
Rakshith R
04670a8b4d doc: add steps for upgrading from 3.10 to 3.11
This commit adds steps for upgrading from 3.10
to 3.11.

Signed-off-by: Rakshith R <rar@redhat.com>
2024-04-02 08:07:30 +00:00
Rakshith R
0e4991ac8b doc: update README for v3.11.0 release
This commit updates README to deprecate 3.9.x release,
add 3.10.1/2 release versions and pin latest release
to v3.11.0

Signed-off-by: Rakshith R <rar@redhat.com>
2024-04-02 08:07:30 +00:00
Praveen M
3c8ea475ec doc: csi driver object options
Signed-off-by: Praveen M <m.praveen@ibm.com>
2024-04-01 09:27:01 +00:00
Niels de Vos
8e57efad98 cleanup: use standard Golang "slices" instead of k8s package
The "slices" package has been introduced in Go 1.21 and can be used
instead of the Kubernetes package that will be replaced by the standard
package at one point too.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-03-29 12:42:24 +00:00
Praveen M
33a888f9ec helm: fix seLinuxMount option for csi driver
This commit fixes the typo from `.Values.seLinuxMount` to
`.Values.CSIDriver.seLinuxMount` used in helm charts.

Signed-off-by: Praveen M <m.praveen@ibm.com>
2024-03-29 10:46:18 +00:00
Niels de Vos
3df396e6f1 rbd: add extra logging while cleaning up snapshots
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-03-28 11:54:28 +00:00
Niels de Vos
ba05c0f5f1 cleanup: reformat generateVolFromSnap() to rbdSnapshot.toVolume()
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-03-28 11:54:28 +00:00
Niels de Vos
a517290ea7 rbd: let parseVolCreateRequest() return a connected rbdVolume
By returning a connected rbdVolume in parseVolCreateRequest(), the
CreateVolume() function can be simplified a little. There is no need to
call the additional Connect() and detect failures with it.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-03-28 11:54:28 +00:00
Niels de Vos
7b2b125b18 rbd: free snapshot resources after allocation
Not all snapshot objects are free'd correctly after they were allocated.
It is possible that some connections to the Ceph cluster were never
closed. This does not need to be a noticeable problem, as connections
are re-used where possible, but it isn't clean either.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-03-28 11:54:28 +00:00
Niels de Vos
18162c71bc cleanup: do not pass an empty snapshot to genSnapFromSnapID()
Just like GenVolFromVolID() the genSnapFromSnapID() function can return
a snapshot. There is no need to allocated an empty snapshot and pass
that to the genSnapFromSnapID() function.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-03-28 11:54:28 +00:00
parth-gr
063319f6e5 rbd: make pool optional in rbd sc if topologyconstraints are present
if rbd storage class is created with topologyconstraintspools
replicated pool was still mandatory, making the pool optional if the
topologyconstraintspools is requested

Closes: https://github.com/ceph/ceph-csi/issues/4380

Signed-off-by: parth-gr <partharora1010@gmail.com>
2024-03-22 13:15:50 +00:00
NymanRobin
5224d58c13 cephfs: add support for encryption in ceph-csi-cephfs chart
the chart currently lacks access to configmap and secrets
this causes the mounting of encrypted file systems to fail

Signed-off-by: NymanRobin <nyman.robin@gmail.com>
2024-03-21 14:58:33 +00:00
Ruslan Khizhnyak
d56c9abbce helm: CSIDriver add labels and seLinuxMount disabling method
Signed-off-by: Ruslan Khizhnyak <rkhizhnyak@ptsecurity.com>
2024-03-21 10:07:23 +00:00
Madhu Rajanna
cd18490451 rebase: update go-ceph to latest commit
updating go-ceph to latest commit to
pull the changes required for ceph fs
quiesce. This is also updating aws
sdk dependency.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-03-21 06:16:42 +00:00
Madhu Rajanna
aa2094ba30 cephfs: unit test for validateVolumeGroupSnapshotRequest
Added unit test for
validateVolumeGroupSnapshotRequest API which
validates the input VolumeGroupSnapshotRequest
request

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-03-21 06:16:42 +00:00
Madhu Rajanna
728a7f5ac7 util: add UnimplementedGroupControllerServer
adding UnimplementedGroupControllerServer to
the DefaultControllerServer struct to avoid
build errors when some non mandatory RPC's
are not implemented.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-03-21 06:16:42 +00:00
Madhu Rajanna
0f724480f5 cephfs: implement DeleteVolumeGroupSnapshot RPC
implemented DeleteVolumeGroupSnapshot RPC which
does below operations

* Basic request validation
* Get the snapshotId's and volumeId's
mapping reserved for the UUID
* Delete snapshot and remove its mapping
from the omap
* Repeat above steps until all the mapping
are removed
* Remove the reserved uuid from the omap
* Reset the filesystem quiesce, This might be
required as cephfs doesnt provide any options to
remove the quiesce, if we get any request with same
ID again we can reuse the quiesce API for same set-id
* Return success if the received error is
Pool not found or key not found.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-03-21 06:16:42 +00:00
Madhu Rajanna
df770e4139 cephfs: implement CreateVolumeGroupSnapshot RPC
implemented CreateVolumeGroupSnapshot RPC which
does below operations

* Basic request validation
* Reserve the UUID for the group name
* Quiesce the filesystem for all the subvolumes
from the input volumeId's
* Take the snapshot for all the input volumeId's
* Add the mapping between volumeId's and snapshot
Id's in omap
* Release the quiesce for the filesystem for
all the subvolumes from the input volumeId's

Undo all the operations if anything fails.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-03-21 06:16:42 +00:00
Madhu Rajanna
ffb2b1144d cephfs: add helper for group options
volumegroup.go holders all the helpers
to extra the group details from the request
and also to extra group details from the
groupID.

This also provide helpers to reserve group
for the request Name and also an undo function
incase if somethings goes wrong and we need to
cleanup the reserved omap entries.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-03-21 06:16:42 +00:00
Madhu Rajanna
86bf74bb5c cephfs: add helper function to getVolumeOptions
added helper function to extract basic
details from the parameters related to volume
options.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-03-21 06:16:42 +00:00
Madhu Rajanna
6a4c45deeb cephfs: add helper for quiesce api
added helper function which calls
the go-ceph API for the quiesce
operations.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-03-21 06:16:42 +00:00
Madhu Rajanna
eff0fe3a23 cephfs: add error for quiesce operation
added ErrInProgress to indicate the
the quiesce operation is in progress.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-03-21 06:16:42 +00:00
Madhu Rajanna
ef25a816a7 cephfs: add locks for volumegroup
Adding a lock for the volumegroup so
that we can take care of serializing
the same requests to ensure same requests
are not served in parallel.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-03-21 06:16:42 +00:00
Madhu Rajanna
b30da094b0 build: add ceph_preview to GO_TAGS_LIST
added required ceph_preview tag to the
GO_TAGS_LIST in Makefile which is
required for FSQuiesce API.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-03-21 06:16:42 +00:00
Madhu Rajanna
6ec86879e6 cephfs: register group controller
register the group controller service
for the cephfs.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-03-21 06:16:42 +00:00
Madhu Rajanna
ff6eda0de1 cephfs: initialize VolumeGroupJournal
initialize VolumeGroupJournal which is
required for volumegroup rados communication

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-03-21 06:16:42 +00:00
Madhu Rajanna
445de7926d cephfs: add validateCreateVolumeGroupSnapshotRequest
added validateCreateVolumeGroupSnapshotRequest
to validate the CreateVolumeGroupSnapshotRequest
request and ensure that all the requirement
options are set. if not, reject the RPC request.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-03-21 06:16:42 +00:00
Madhu Rajanna
68e93a31cc journal: fix connection problem with groupjournal
Same group jounral config need to be reused
for multiple connection where different monitors
and users are used, for that reason create a unique
connection each time.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-03-21 06:16:42 +00:00