Commit Graph

541 Commits

Author SHA1 Message Date
Niels de Vos
f87d06ed85 build: move e2e dependencies into e2e/go.mod
Several packages are only used while running the e2e suite. These
packages are less important to update, as the they can not influence the
final executable that is part of the Ceph-CSI container-image.

By moving these dependencies out of the main Ceph-CSI go.mod, it is
easier to identify if a reported CVE affects Ceph-CSI, or only the
testing (like most of the Kubernetes CVEs).

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2025-03-04 17:43:49 +01:00
Niels de Vos
9bd6cbf5ca util: move kernel version functions to pkg/util/kernel
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2025-03-04 13:58:41 +01:00
Niels de Vos
6ea73af34c util: move EncryptionType(s) to pkg/util/crypto
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2025-03-04 13:58:41 +01:00
Yite Gu
7595e20969 rbd: support QoS based on capacity for rbd volume
1. QoS provides settings for rbd volume read/write iops
   and read/write bandwidth.
2. All QoS parameters are placed in the SC,
   send QoS parameters from SC to Cephcsi through PVC create request.
3. We need provide QoS parameters in the SC as below:
   - BaseReadIops
   - BaseWriteIops
   - BaseReadBytesPerSecond
   - BaseWriteBytesPerSecond
   - ReadIopsPerGB
   - WriteIopsPerGB
   - ReadBpsPerGB
   - WriteBpsPerGB
   - BaseVolSizeBytes
   There are 4 base qos parameters among them, when users apply for
   a volume capacity equal to or less than BaseVolSizebytes, use base
   qos limit. For the portion of capacity exceeding BaseVolSizebytes,
   QoS will be increased in steps set per GB. If the step size parameter
   per GB is not provided, only base QoS limit will be used and not associated
   with capacity size.
4. If PVC has resize request, adjust the QoS limit
   according to the QoS parameters after resizing.

Signed-off-by: Yite Gu <guyite@bytedance.com>
2025-02-17 18:25:33 +00:00
Niels de Vos
4d1548c481 e2e: skip "Resize PVC and check application directory size" for NFS
The resize test for NFS seems to fail with the following error:

    expected size 1Gi found 35G

This might be because there is a new Ceph release, this needs furtner
investigation.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2025-02-12 14:33:25 +00:00
Niraj Yadav
477d2ad517 e2e: Note to update to userID and userKey
Once the version we use for upgrade testing does
not depend on adminID and adminKey we should update
the tests to use just the userID and userKey.

Signed-off-by: Niraj Yadav <niryadav@redhat.com>
2025-01-08 13:48:36 +00:00
Praveen M
54a8b50957 ci: non-constant format string (govet)
Signed-off-by: Praveen M <m.praveen@ibm.com>
2025-01-08 11:56:24 +00:00
Praveen M
dfc9bdaf03 ci: address shadowing of predeclared identifier
- Comparing integers makes more sense than comparing the strings.

Signed-off-by: Praveen M <m.praveen@ibm.com>
2025-01-08 11:56:24 +00:00
Praveen M
ea205410f5 ci: update golangci-lint to v1.62.2
- gomnd is replaced by mnd in v1.58.0
- gosec exlcude G115 rule (Potential integer overflow when converting between integer types)
- disable new iface linter
- disable new recvcheck linter

Signed-off-by: Praveen M <m.praveen@ibm.com>
2025-01-08 11:56:24 +00:00
Rakshith R
afc08c531c e2e: test creation of additional groupSnaps to test minSnapLimit
Signed-off-by: Rakshith R <rar@redhat.com>
2024-12-17 15:15:42 +00:00
Madhu Rajanna
a88f86e9a4 e2e: remove vsg flag present in older snapshotter
Remove the vsg flag present in the older snapshotter
which is meant only for the Alpha Release and we are
no longer installing Alpha CRD.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-12-17 10:55:39 +00:00
Madhu Rajanna
a17fef07e6 e2e: update snapshot to v1beta1
updating the snapshot client and
test to use v1beta1 client and API.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-12-17 10:55:39 +00:00
Praveen M
88b7e0d668 e2e: validate images in trash for volumegroupsnapshot test
Signed-off-by: Praveen M <m.praveen@ibm.com>
2024-12-16 13:36:22 +00:00
Oded Viner
dd1c302bd5 rbd: added rbd info to validateRBDImageCount func
Signed-off-by: Oded Viner <oviner@redhat.com>
2024-11-22 15:33:40 +00:00
Niels de Vos
c451997762 e2e: detect support for VolumeGroupSnapshot in librbd
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-11-04 15:38:18 +00:00
Niels de Vos
b59a701777 e2e: add tests for RBD VolumeGroupSnapshots
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-11-04 15:38:18 +00:00
Madhu Rajanna
f4f03044f2 e2e: enhance volumegroupsnapshot test
update the e2e for volumegroupsnapshot
to create application pods from clone pvc
and delete the pods once we are able to
create the pods.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-11-04 10:28:27 +00:00
Madhu Rajanna
00c7fb3136 e2e: log stderror in execCommandInPodAndAllowFail
If the command we try to execute fails, log
the stdError along with error for better debugging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-10-29 17:30:49 +00:00
Madhu Rajanna
98471b27fc e2e: add a test case of ROX PVC-PVC clone
adding a test case to test below case for RBD

* Create PVC and application
* Scale down application pod
* Create PVC-PVC ROX clone
* Try to create many pods that use the ROX PVC
* Try to write the data and verify only read
access.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-10-29 17:30:49 +00:00
Praveen M
3aa91cfa6f e2e: cephfs rados namespace test
Signed-off-by: Praveen M <m.praveen@ibm.com>
2024-10-21 14:11:27 +00:00
Madhu Rajanna
d2ddd52151 e2e: add framework for volumegroupsnapshot
adding a framework to test the volumegroupsnapshot
for both cephfs and rbd and added a test case
for cephfs.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-09-12 13:07:13 +00:00
Madhu Rajanna
744b8e1c1c ci: log pvc events
incase if the pvc is not bound
yet, log the pvc events to
check what is happening.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-09-12 13:07:13 +00:00
Madhu Rajanna
5c621d2e82 e2e: set log level to 5 in csi pods
all the sidecar log level is not set
to 5 by default, without debug logs
we cannot understand what is wrong
in the tests, this commits sets the
log level to 5 for csi pods.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-09-12 13:07:13 +00:00
Madhu Rajanna
c48f5bff37 rebase: update external-snapshotter to v8
updating external-snapshotter to v8
release.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-09-12 13:07:13 +00:00
Praveen M
202f43c82d e2e: remove enableTopologyInTemplate method
This commit removes `enableTopologyInTemplate` method as this is not
needed anymore.

Signed-off-by: Praveen M <m.praveen@ibm.com>
2024-06-13 10:08:15 +00:00
Niels de Vos
03413a53fd e2e: retry when a pod does not have a host assigned (yet)
Errors like "pod nfs-820 does not have a host assigned" seem to get
reported when a Pod is not completely started yet, or was restarted
while trying to access it.

Reported: https://github.com/ceph/ceph-csi/pull/4656#issuecomment-2151794926
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-06-06 19:23:15 +00:00
Madhu Rajanna
24112917ec e2e: fix logging as per new controller runtime
fixing logging as per new controller runtime
errors

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-05-24 09:27:53 +00:00
Niels de Vos
e85914fc0d build: address 'intrange' linter warning
golangci-lint warns about this:

    for loop can be changed to use an integer range (Go 1.22+)
    (intrange)

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-04-29 14:36:48 +00:00
Niels de Vos
0e7b06e9d0 build: address 'copyloopvar' linter warning
golangci-lint reports these:

    The copy of the 'for' variable "kmsID" can be deleted (Go 1.22+)
    (copyloopvar)

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-04-29 14:36:48 +00:00
Rakshith R
1bb78fdf43 e2e: validate PVC-PVC clone creation with deleted parent snap
This commit modifies a test case to check creation of
PVC-PVC clone of a restored PVC when parent snapshot
is deleted.

Signed-off-by: Rakshith R <rar@redhat.com>
2024-04-23 12:04:59 +00:00
Madhu Rajanna
6961b103b8 e2e: address golangci-lint issues
addessed golangci-lint issues in
e2e folder.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-04-09 06:55:23 +00:00
Madhu Rajanna
1e0254daba util: use slices package for contains
use slices package to check whether
the slice contains the element or not.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-04-02 14:10:11 +00:00
Niels de Vos
86b5db90bc cleanup: rename EncryptionTypeString() to EncryptionType.String()
This makes it easier to log the EncryptionType as string, or int,
whatever is preferred. Standard fmt formatting notations like %s or %d
can be used now.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-04-02 14:04:10 +00:00
parth-gr
063319f6e5 rbd: make pool optional in rbd sc if topologyconstraints are present
if rbd storage class is created with topologyconstraintspools
replicated pool was still mandatory, making the pool optional if the
topologyconstraintspools is requested

Closes: https://github.com/ceph/ceph-csi/issues/4380

Signed-off-by: parth-gr <partharora1010@gmail.com>
2024-03-22 13:15:50 +00:00
Niels de Vos
a469a66f9d e2e: replace quoted namespace in templates too
Some templates are now generated with the API, and these include
namespaces as "quotes" values. Namespace replacing in the templates need
to replace both the unquoted and quoted strings.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-03-15 08:02:25 +00:00
Madhu Rajanna
a45ad5be9f rebase: update external-snapshotter client to v7
updating external-snapshotter client to
use the latest release

updates #4435

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-02-20 10:18:11 +00:00
Madhu Rajanna
e008e16095 e2e: add e2e to test static pvc
added e2e test case to verify static
pvc with and without fsName in the PV
object.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-01-12 18:27:45 +00:00
Praveen M
50e505619c deploy: added json field tags for csi config map
This commit adds the json field tags for csi config map for
encoding and decoding JSON.

Signed-off-by: Praveen M <m.praveen@ibm.com>
2023-12-21 17:44:46 +00:00
Niels de Vos
35f034f156 e2e: update to Kubernetes 1.29 api
Signed-off-by: Niels de Vos <niels@nixpanic.net>
2023-12-21 13:34:39 +00:00
Praveen M
6f9b1ab04d cephfs: remove subvolumegroup creation
Signed-off-by: Praveen M <m.praveen@ibm.com>
2023-11-10 11:41:35 +00:00
Praveen M
c4e373c72f deploy: support for read affinity options per cluster
Implemented the capability to include read affinity options
for individual clusters within the ceph-csi-config ConfigMap.
This allows users to configure the crush location for each
cluster separately. The read affinity options specified in
the ConfigMap will supersede those provided via command line arguments.

Signed-off-by: Praveen M <m.praveen@ibm.com>
2023-11-08 21:17:00 +00:00
Madhu Rajanna
304462c7cc cleanup: fix spellcheck errors
fixed spellcheck errors caught in
CI.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2023-11-08 12:32:06 +00:00
Madhu Rajanna
55768007f4 e2e: remove multiple labels to node
update the e2e code to remove multiple
labels to the node at a time.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2023-11-03 18:24:54 +00:00
Madhu Rajanna
a9e8a32a98 e2e: add multiple labels to node
update the e2e code to add multiple
labels to the node at a time.

fixes: #4146

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2023-11-03 18:24:54 +00:00
karthik-us
5ff0607360 e2e: To test 500MB PVC creation
Adding e2e test to check for successful PVC creation
of 500MB.

Signed-off-by: karthik-us <ksubrahm@redhat.com>
2023-10-12 17:31:46 +00:00
Praveen M
6719d6497f e2e: added test to verify read affinity functionality
e2e test case is added to test if read affinity is enabled by
verifying read_from_replica=localize option is passed

Signed-off-by: Praveen M <m.praveen@ibm.com>
2023-09-26 07:02:21 +00:00
Madhu Rajanna
9809f365fc e2e: add e2e test for RWX from ROX
added an e2e test case to create
RWX clone from ROX and also try
to write extra data in the RWX
cloned PVC.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2023-08-31 11:51:18 +00:00
Madhu Rajanna
10bec8a002 ci: skip enabling nfs modules in e2e
As we dont need to enable nfs modules
in ceph v16.2.8 onwards skipping this one.

Because of this one we have a regression
in nfs export

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2023-08-25 17:50:40 +00:00
Madhu Rajanna
e76e2f88b1 e2e: add --yes-i-really-mean-it for pool creation
in recent ceph version .nfs pool creation
is failing, as we are sure about creating the
pools in the e2e tests, tring to create the pool
with required extra agruments to make it successful.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2023-08-25 17:50:40 +00:00
riya-singhal31
d435b59656 e2e: add timeout for pvc deletion in ephemeral e2e
this commit adds the timeout to wait for pvc
deletion after the deletion of pod in test.

Signed-off-by: riya-singhal31 <rsinghal@redhat.com>
2023-08-25 14:22:00 +00:00