Commit Graph

334 Commits

Author SHA1 Message Date
Niels de Vos
460073521b ci: prepare Jenkins jobs for Kubernetes 1.30
Kubernetes 1.30 is in Alpha state, it should be usable for testing
Ceph-CSI against now.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-02-22 13:14:07 +01:00
Niels de Vos
05f13b2b82 ci: CentOS CI networking is stable again, remove pull-loop
The for loop in Groovy does not work, but fortunately it seems that the
CentOS CI network environment is stable again. Pulling container images
is working as it did before.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-01-31 10:51:09 +01:00
Niels de Vos
203b377137 ci: correct the format of pipeline-scm.scm
It seems the `git` component of `pipeline-scm.scm` needs to be a list.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-01-31 10:33:57 +01:00
Niels de Vos
9273955d92 ci: replace inline dsl: script by Groovy files
Jenkins Jobs Builder does not like the inline `dsl:` scripts anymore, so
replace them by Groovy files in the main directory, similar to other
jobs.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-01-31 09:54:41 +01:00
Niels de Vos
6e38694a11 ci: the order of --allow-empty-variables for jenkins-jobs is important
Jenkins Jobs Builder fails like this, indicating the order of the
parameters is important:

```
$ jenkins-jobs update --allow-empty-variables --delete-old jobs
usage: jenkins-jobs [-h] [--conf CONF] [-l LOG_LEVEL] [--ignore-cache]
                    [--flush-cache] [--version] [--allow-empty-variables]
                    [--server SECTION] [--user USER] [--password PASSWORD]
                    {delete,delete-all,get-plugins-info,list,test,update} ...
jenkins-jobs: error: unrecognized arguments: --allow-empty-variables
```

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-01-30 18:14:58 +01:00
Niels de Vos
6baa675fa0 ci: allow missing RUN_DISPLAY_URL variable in Jenkins Jobs
The `RUN_DISPLAY_URL` is automatically set by Jenkins, but it seems the
latest Jenkins Jobs Builder update does not detect it automatically.
Run the JJB commands with `--allow-empty-variables`.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-01-30 18:08:18 +01:00
Niels de Vos
ad0a645324 ci: move inline script into separate build-images.groovy file
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-01-30 17:43:25 +01:00
Niels de Vos
31cddd1f36 ci: define i as type int in for loop
The output of the for loop seemt to be `groovy.lang.IntRange`, which is
not very useful. Maybe it is required to define the variable of type
`int`?

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-01-30 17:43:25 +01:00
Niels de Vos
97afd83f2b ci: retry pulling container images up to 10x
There seems to be some failing network component in the CI environment
that prevents pulling container images. Even pulling it from the
registry to my laptop fails often, but with a few retries everything
gets pulled eventually.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-01-30 15:34:09 +00:00
Riya Singhal
e61b2f4aaf rebase: update jobs for kubernetes 1.29
add support to run kubernetes 1.29 jobs

Signed-off-by: Riya Singhal <rsinghal@redhat.com>
2024-01-04 10:09:30 +00:00
karthik-us
ea0464958c ci: Fix helm test E2E_ARGS
Fix the helm tests E2E_ARGS interpolation.

Signed-off-by: karthik-us <ksubrahm@redhat.com>
2023-11-08 11:37:52 +00:00
karthik-us
1245b49cb9 ci: Fix E2E_ARGS for helm test
The test_type and deployment related args are being passed
as make option instead of the E2E_ARGS. Pass it as E2E_ARGS.

Signed-off-by: karthik-us <ksubrahm@redhat.com>
2023-11-08 06:09:40 +00:00
karthik-us
b8c16ccbac ci: Pass test_type as E2E_ARGS
The test_type is being passed as the make option instead
of the E2E_ARGS. This fix should pass it as the E2E_ARGS.

Signed-off-by: karthik-us <ksubrahm@redhat.com>
2023-11-06 11:13:28 +00:00
karthik-us
c659487084 ci: make test_type optional
Make test_type parameter optional to keep the current test run
behavior intact.

Signed-off-by: karthik-us <ksubrahm@redhat.com>
2023-11-06 06:57:20 +00:00
karthik-us
674b1e811a ci: add capability to run specific e2e tests
The current e2e test runs will trigger all 3 types of
tests which takes a lot of time to complete. If the
patch impacta only a single type, then having an option
to trigger only that particular e2e test can save
considerable amount of time based on the test type.

This can also help in reducing the time in testing when
a patch failed on a particular e2e test type and triggering
only that e2e with the fix for the failure or re-trigger
only that test again to check for false positives.

Signed-off-by: karthik-us <ksubrahm@redhat.com>
2023-11-06 06:57:20 +00:00
Madhu Rajanna
a6db0d99cd ci: add sudo prefix for status commands
few commands are getting permission denied error
adding sudo is fixing the problem

```
Warning: Permanently added '192.168.49.2' (ECDSA) to the
list of known hosts.
Hint: You are currently not seeing messages from other
users and the system.
Users in the 'systemd-journal' group can see all messages.
Pass -q to turn off this notice.
No journal files were opened due to insufficient permissions.
```
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2023-08-24 11:14:11 +00:00
Madhu Rajanna
932c7c5879 rebase: use kubernetes 1.27 in upgrade jobs
update kubernetes version to v1.27 in upgrade
jobs.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2023-08-17 11:17:52 +00:00
Madhu Rajanna
fe2043e66f rebase: update jobs for kubernetes 1.28
add support to run kubernetes 1.28 jobs

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2023-08-17 11:17:52 +00:00
Niels de Vos
274739f70b ci: strip quay.io from mirrored ceph/ceph:v18 image
CI jobs pull the ceph/ceph:v18 image (without `quay.io`). If the
mirroring includes the registry host, the image can not be found.

Updates: #4030
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2023-08-04 07:35:10 +00:00
Madhu Rajanna
32335dc9eb ci: update images for mirroring
As we need to move to latest Rook
and Ceph releases in devel branch
updating mirroring images to push
latest images to internal registry.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2023-08-02 20:29:31 +00:00
Rakshith R
ae5e455322 ci: override csi_upgrade_version from build.env
This commit enables overriding csi_upgrade_version
used for upgrade testing with CSI_UPGRADE_VERSION
value from build.env.

Signed-off-by: Rakshith R <rar@redhat.com>
2023-07-21 10:57:40 +00:00
karthik-us
2ab3e8337f ci: Update csi_upgrade_version
Point the csi_upgrade_version from v3.7.2 to the
latest release v3.9.0.

Updates: #3911
Signed-off-by: karthik-us <ksubrahm@redhat.com>
2023-07-13 08:12:53 +00:00
karthik-us
d14bba7c12 doc: Update/remove broken links
Updates few broken links, removes a non existent link and some nit-picks.

Signed-off-by: karthik-us <ksubrahm@redhat.com>
2023-07-04 14:58:17 +00:00
Niels de Vos
ab521dba96 ci: fix trigger command for ci/centos/mini-e2e-helm jobs
The `/test ci/centos/mini-e2e-helm` has been broken with the last change
where `/test all` was removed. A remaining '(' makes the regular
expression invalid, and jobs fail to get started.

Fixes: f9f3bb9 ("ci: remove `/retest all` command for Jenkins jobs")
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2023-07-03 15:26:37 +00:00
Niels de Vos
f9f3bb90a5 ci: remove /retest all command for Jenkins jobs
`/retest all` causes a spike in resource consumption in Jenkins and the
OpenShift cluster kills the Pod. That means tests are not fully running
yet, and results never arrive back in the PR. Instead of `/retest all`,
the `ok-to-test` label can be used to trigger required tests with a
slight delay between each command.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2023-06-29 11:10:48 +00:00
Niels de Vos
5e728f51b2 build: remove urllib3 downgrade in jjb container, use timeout instead
The upstream reported issue suggests adding a `timeout` value in the
Jenkins Jobs Builder configuration. This looks like a nicer workaround
than downgrading urllib3.

Fixes: #3946
See-also: https://storyboard.openstack.org/#!/story/2010752
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2023-06-28 12:48:12 +00:00
Niels de Vos
3592bbedbb build: use urllib3 < 2 with Jenkins jobs Builder
jenkins-job-builder does not seem to work with urllib3 2.x, force using
version 1.x as a workaround.

Updates: #3946
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2023-06-27 12:51:26 +00:00
Rakshith R
a6ac81133f ci: remove k8s 1.24 support
K8s 1.24 will be End of Life on 2023-07-28.
Therefore, this commit removes support
for k8s 1.24.

refer:
https://kubernetes.io/releases/#release-v1-24

Signed-off-by: Rakshith R <rar@redhat.com>
2023-06-21 10:53:42 +00:00
Niels de Vos
a02b38ac27 ci: use CentOS Stream 9 for Jenkins Jobs Builder image
CentOS Stream 8 does not provide a module named 'dataclasses'. It seems
current versions of Jenkins Jobs Builder require that. Python 3.7
offers the module by default, so using a new base distribution with
Python 3.9 has the module too.

See-also: #3909
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2023-06-21 08:11:51 +00:00
Madhu Rajanna
f7942edc6c ci: remove kubernetes 1.23 support
removed ability to run tests with
kubernetes 1.23 as its no longer
maintained version.

fixes: #3636

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2023-04-18 10:05:26 +02:00
Niels de Vos
cca994eb1a ci: run registry garbage collection every week
In the old OpenShift cluster the registry consumer 800+ GB of data. Once
running the garbage collection manually, the consumption reduced to a
little over 8GB. Let's be nice users of the infrastructure and run
garbage collection weekly.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2023-03-20 14:20:53 +00:00
Niels de Vos
8181d13313 ci: update mirroring container-images job for new OCP
The new OpenShift cluster complained about deprecated types and missing
options in the yaml files.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2023-03-20 10:52:12 +00:00
Niels de Vos
b0e9eeba36 ci: correct parameter passing in checkout-repo.sh
ShellCheck failed with the following warning:

```
SC2086 (info): Double quote to prevent globbing and word splitting.

Did you mean:
git config --global --add safe.directory "${PWD}"
```

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2023-03-20 09:21:41 +00:00
Niels de Vos
8135680065 ci: migrate to new OpenShift cluster running on AWS
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2023-03-20 09:21:41 +00:00
Niels de Vos
234471af33 ci: allow /opt/build for git repository when deploying jobs
When Jenkins Jobs have been modified, they should get deployed in the
Jenkins environment. This seems to fail with the following error:

```
Initialized empty Git repository in /opt/build/.git/
fatal: detected dubious ownership in repository at '/opt/build'
To add an exception for this directory, call:

	git config --global --add safe.directory /opt/build
```

By marking /opt/build as a safe directory in the global git
configuration file, this problem should not occur anymore.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2023-01-30 15:15:06 +01:00
Niels de Vos
f7ba14b2ed ci: add jobs for running with Kubernetes 1.27
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2023-01-30 09:48:56 +00:00
Humble Chirammal
4037ebca49 ci: update rook version to 1.10.9
this commit update ci rook version to latest.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2023-01-13 09:50:28 +00:00
Niels de Vos
b557fc82bd ci: move ci-job-validation to duffy API
It seems all jobs were converted to the new `duffy` command, except
ci-job-validation which still used the old `cico` command.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2023-01-13 09:46:26 +00:00
Niels de Vos
3880d642b9 ci: use new duffy command for CentOS CI
The `cico` command is getting deprecated, and the new `duffy` command
should be used instead. The new command requires a `~/.config/duffy`
file that contains an API endpoint and credentials.

See-also: https://sigs.centos.org/guide/ci/#installing-and-configuring-duffy-client
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-12-12 08:23:30 +00:00
Madhu Rajanna
69329b5592 ci: use kubernetes 1.25 for upgrade tests
Upgrade tests always runs with the latest
release of kubernetes, updating the
kubernetes to latest available release
in the CI.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-10-28 19:49:45 +00:00
Madhu Rajanna
253b6a9d2b ci: remove kubernetes 1.22 from CI
As we dont longer required to test with
Kubernetes 1.22 removing it from the CI.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-10-28 19:49:45 +00:00
Madhu Rajanna
65411d7975 ci: update outdates kubernetes version
use latest used kubernetes version or supported
kubernetes version for upgrade testing.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-10-26 10:18:14 +00:00
Madhu Rajanna
8cabc0327f ci: run upgrade tests with latest release
Run E2E tests with latest available release.

fixes: #2818

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-10-26 10:18:14 +00:00
Madhu Rajanna
c1255f809b ci: disable e2e tests by default in CI
Disable e2e tests by default in CI to
save resources. cephcsi maintainers or
contributors can add the ok-to-test label
to trigger the e2e tests.

depends on #3468

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-10-26 04:43:41 +00:00
Madhu Rajanna
d6b721b7a8 ci: use rook v1.10.4 in CI
update Rook image version to
v1.10.4 which is latest release

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-10-24 08:48:07 +00:00
Niels de Vos
83369a36f1 ci: install wget for downloading Helm
Installing Helm fails if `wget` is not available. The script that
installs Helm does not seem to abort though.

By installing `wget` in preparation for deploying with Helm, the CI jobs
should get ready to run successfully.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-10-13 06:26:56 +00:00
Niels de Vos
cf85580c16 ci: include journalctl --boot from the host in system status
Now that we run with Podman, the journal from the container that was
started by minikube is not very useful. In addition to the journal from
the minikube container or VM, include the journal from the host.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-10-12 16:36:15 +00:00
Niels de Vos
fce1892115 ci: copy images into minikube container when VM_DRIVER=podman
Even when minikube is running with `--driver=podman`, `minikube ssh`
works and container images can be copied into the minikube container.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-10-12 10:14:49 +00:00
Niels de Vos
893da82f71 ci: use VM_DRIVER=podman for AWS virtual-machines
With the CentOS CI machines moving to AWS EC2 virtual-machines, there is
no option to run minikube with a VM anymore. Instead, run minikube with
the Podman driver and partition the extra xvdb EBS volume into three
pieces.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-10-10 07:18:28 +00:00
Madhu Rajanna
7681ef27ff ci: update rook image in centos ci
update rook image to latest release
in ci

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-10-07 16:11:37 +00:00