Commit Graph

52 Commits

Author SHA1 Message Date
Niels de Vos
918d5e0870 ci: enable running of k8s-e2e-external-storage job by default
The job seems stable, and can be run by default now. Once it has been
run on several PRs, the `ci/centos/k8s-e2e-external-storage` job ID
can be added to the Mergify configuration.

See-also: https://jenkins-ceph-csi.apps.ocp.ci.centos.org/blue/organizations/jenkins/k8s-e2e-external-storage/activity
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-04-26 04:18:39 +00:00
Niels de Vos
b5aa0b11d7 ci: add optional Kubernetes 1.21 job
To test a PR with Kubernetes 1.21, leave a comment in the PR like:

    /test ci/centos/mini-e2e-helm/k8s-1.21

The status of the job will be recorded in the PR, but running this job
is not required (yet).

Updates: #1963
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-04-20 07:08:16 +00:00
Yug
e030097eaa ci: Disable containerized-tests job
The containerized-test job should run only if
manually triggered.

Signed-off-by: Yug <yuggupta27@gmail.com>
2021-04-07 14:16:38 +00:00
Niels de Vos
1c2974d49e ci: the "master" branch got renamed to "devel"
Closes: #1193
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-03-01 05:22:06 +00:00
Niels de Vos
0b78359b9b ci: do not run jobs with Kubernetes v1.18 anymore
Kubernetes v1.20 has been released, so lets use that for testing. Note
that v1.18 is still maintained, but our CI jobs will not consume it
anymore.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-18 06:35:49 +00:00
Niels de Vos
5562a6ded9 ci: add jobs with Kubernetes v1.20
Add CI jobs for Kubernetes v1.20 testing. These jobs will run, but are
not (yet) required before changes get merged.

Updates: #1784
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-15 13:19:04 +00:00
Niels de Vos
629a8e0166 ci: do not run commitlint on "/retest all"
The commitlint job is not used normally, so there is no need to run it
when all CI jobs need retesting.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-17 10:16:46 +00:00
Niels de Vos
2ad602e9f2 ci: fix Blue Ocean status-url for JJB templates
As ${RUN_DISPLAY_URL} also matches the format for template
substitutions, it can not be used like this for all Jenkins Jobs Builder
files. Instead, $RUN_DISPLAY_URL can be used, as it does not have the
{curly_braces} format and the template engine skips it.

Fixes: 87870369 (ci: link to the job in the Blue Ocean webui)
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-04 09:58:46 +00:00
Niels de Vos
8787036903 ci: link to the job in the Blue Ocean webui
The new Blue Ocean webui is much easier to navigate than the traditional
Jenkins webui. Developers that want to check failures of CI jobs will be
able to find the logs much quicker as each stage/step has a clear
success or failed status.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-04 04:51:49 +00:00
Niels de Vos
9c4009bd14 ci: pass BASE_IMAGE to "oc start-build"
Because the parallel steps, the build.env is not in the local directory
where the command is executed. So fetch the BASE_IMAGE before continuing
with 'oc start-build', and pass the name of the image on the
commandline.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-04 04:28:28 +00:00
liranmauda
c281511a11 ci: Add the ability to run JJB from private branch/fork
Add the ability to run JJB from private branch/fork

jjb-deploy.yaml:
- Add GIT_REPO parameter (not required) with default value

jjb.sh:
- Changing PS4 for more indicative debug prompt
- Adding flags to get the repo and branch

jjb-deploy.yaml:
- Calling the jjb.sh with the correct flags

Signed-off-by: liranmauda <liran.mauda@gmail.com>
2020-11-03 07:26:43 +00:00
Niels de Vos
1700f8585c ci: add a build-images job
The new 'build-images' job rebuilds ceph-csi:canary, ceph-csi:devel and
ceph-csi:test images after a PR has been merged in the master branch.
These images can then be used by other CI jobs, to improve the speed by
reducing unneeded rebuilds of the images.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-30 13:07:58 +00:00
Niels de Vos
466ada3860 ci: use github/refs/pull/<pr-id>/merge instead of doing a rebase
When fetching refs/pull/<pr-id/merge from GitHub, there is no need to do
a manual rebase. This makes things easier, as a the scripted rebases
sometimes cause CI jobs to fail.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-28 04:12:48 +00:00
Niels de Vos
cd893ba376 ci: use major Kubernetes versions again
With the update to minikube v1.14.1 downloading binaries for the recent
Kubernetes patch releases works again. Enable detecttion of the latest
patch releases for use in the CI jobs.

Fixes: #1588
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-27 13:40:25 +00:00
Niels de Vos
e103d4a0cc ci: add k8s-e2e-external-storage job
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-23 09:30:14 +00:00
Niels de Vos
86375adf9e ci: run only one jjb-validate at the time
The jjb-validate job creates a Batch Job in OCP. This job has a fixed
name, and fails to get created when one with the same name already
exists. There is no need to run jjb-validate concurrently, the number of
changes for CI jobs are not blocked by this serialization.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-23 07:16:24 +00:00
Niels de Vos
debb54a1dd ci: use Kubernetes 1.18.9 for CI jobs
The latest Kubernetes patch release (v1.18.10) can not be deployed with
minikube. Selecting version 1.18.9 until the problems with minikube are
addressed.

Updates: #1588
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-19 06:42:35 +00:00
Niels de Vos
ea770fb4c8 ci: use Kubernetes 1.19.2 for CI jobs
The latest Kubernetes patch release (v1.19.3) can not be deployed with
minikube. Selecting version 1.19.2 until the problems with minikube are
addressed.

Updates: #1588
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-15 09:28:39 +00:00
Yug
bd247db377 ci: Trigger upgrade test jobs automatically
Since upgrade tests will become a necessary to
merge PRs, it is better to trigger them
automaticaly now.

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-10-01 07:19:17 +00:00
Yug
0895d864e7 ci: update k8s version for upgrade tests
As the job seems to be unstable for k8s v1.19,
reducing the version to achieve stability for
the same.

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-09-25 12:28:34 +00:00
Yug
31769aa7dc ci: trigger upgrade tests job only on phrase
To avoid runs of upgrade tests on all PRs till
the job is stable, trigger it only through
phrase.

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-09-24 15:02:12 +00:00
Yug
118015d762 ci: Fetch csi upgrade version from configuration file
Instead of accessing the upgrade version from environment
variable, set the variable directly in configuration file.

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-09-23 08:53:15 +00:00
Niels de Vos
01de4eacfc ci: restructure 'parameters' section for upgrade-tests
The section containing the paramaters does not seem to be structured
correctly, correct it.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-09-22 10:30:38 +00:00
Niels de Vos
926817fb54 ci: remove unused parameters section
The incomplete 'parameters:' section causes validation and deploy
issues for jobs.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-09-22 10:30:38 +00:00
Niels de Vos
5f9222d7ee ci: run jjb-validate batch job against a PR
The batch job templates for OpenShift use a GIT_REF variable that is
used by the jobs to checkout the repository. This defaults to
"ci/centos", but it never is adjusted when jjb-validate runs against a
PR.

With the new GIT_REF environment variable in the jjb.sh script, the
variable can now be used to checkout a PR from gitHub and run the
validation against that.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-09-22 11:59:31 +02:00
Niels de Vos
fef7bfdf23 ci: do not set a ghprbPullId parameter
The GitHub Pull Requests Builder plugin sets the ghprbPullId parameter.
There is no need to make it configurable, and it even overloads the
value from the plugin by the default value.

This cauesed the jjb-validate job to run against the ci/centos branch
for each PR, not against the contents of the PR itself.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-09-22 11:59:31 +02:00
Niels de Vos
79225080f9 ci: sandbox is not a supported option
The Jenkins Jobs Builder files reference 'sandbox: true' as an option,
but this does not exist (anymore). Having the option causes warnings in
the Jenkins WebUI for admin users.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-09-22 11:59:31 +02:00
Niels de Vos
cb4fd5338b ci: correct referencing template for upgrade-tests jobs
Fixes: #1489
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-09-22 11:59:31 +02:00
Yug
2e2df0a877 ci: Add template file for upgrade-tests job
This template file contains the configuration
parameters for the upgrade-tests job.

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-09-22 07:25:00 +00:00
Niels de Vos
e353d406eb ci: pass major k8s version to jobs, detect patch release
Jobs can now pass the wanted Kubernetes major version (like '1.19') to
the Jenkins Pipeline scripts. The Pipelines detect the most recent patch
release for the major version with the new get_patch_release.py script.

This causes the CI Job status context to not have the patch number (last
digit of the release) included anymore. Restarting a test will only need
the major version number, as does updating the Mergify configuration.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-09-15 17:02:04 +05:30
Madhu Rajanna
8fd212a797 ci: update kubernetes version for e2e job
removed 1.17 as we are testing only two versions
in centos CI. as v1.19 is released, we will be
testing v1.18 and v1.19 in CI.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-09-09 14:02:45 +00:00
Niels de Vos
5830a2f1bf ci: use GitHub Hooks to trigger jobs
By using GitHub Hooks, jobs get triggers quickers. It also may prevent
the occasional hickup in the Jenkins GitHub-Pull-Request-Builder while
polling events for the cron-jobs.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-08-17 09:45:07 +00:00
Niels de Vos
648c3f7f77 ci: allow everyone to run the "commitlint" job
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-08-10 13:04:23 +00:00
Niels de Vos
fdb17ace76 ci: make members of the GitHub "ceph" organization admins
Not everyone can restart CI jobs, only members of the Ceph organization
in GitHub. All jibs (except "commitlint") are started automatically, so
there is no functional change.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-08-10 12:02:13 +00:00
Niels de Vos
65a0e5c40b ci: do not run commitlint for admin user/group
commitlint should not run automatically on PRs, regardless who posted
the PR. Everyone should be able to trigger a run of the job with `/test
commitlint` anyway.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-08-07 14:15:51 +00:00
Niels de Vos
2e2f69e04a ci: run commitlint when requested, not containeriized-tests
The jobs/commitlint.yaml refered to the containerized-test.groovy
script, not the commitlint.groovy script.

Make sure to run the righ job when requested. Add some comments to make
things a little clearer.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-08-07 14:15:51 +00:00
Niels de Vos
120bbc66ab ci: add commitlint fallback job
Each PR gets tested by the commitlint app (which is deprecated). On
occasion the app does not pickup the change in a PR, and the only way to
start a retest is to re-push the PR. All tests will then get executed
again, even if only commitlint was needed.

The new commitlint CI job introduces '/retest commitlint' and runs in
the CentOS CI like other Jenkins jobs. However, the job does not start
automatically on each (updated) PR, but needs manual starting with the
'/test commitlint' or '/retest commitlint' comments. The normal way of
running commitlint is with the app (at least for now).

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-08-07 12:52:05 +00:00
Niels de Vos
020d3eb212 ci: add mini-e2e-helm Jenkins job
The mini-e2e-helm.groovy script is mostly a copy of mini-e2e.groovy that
does the deployment through the e2e.test executable. The new script
installs Helm and deploys Ceph-CSI through that.

Once the e2e tests have successfully finished, the deployment of
Ceph-CSI is removed again.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-08-05 05:36:49 +00:00
Niels de Vos
0f811e9735 ci: jjb-deploy should have a "deployment" stage
The stage is called "validate" at the moment, this is a copy/paste
mistake from the jjb-validate job.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-08-03 15:36:47 +00:00
Niels de Vos
86448395a5 ci: job name should not contain a "/"
The "/" in the name may have a conflict with the existing "mini-e2e" job
name. Rename the new generated jobs to "mini-e2e_k8s-1.x.y", with a "_"
instead of "/".

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-08-03 16:58:15 +02:00
Niels de Vos
322698a313 ci: pass ghprbPullId as parameter to jjb-validate job
Currently the jjb-validate job always checks out the ci/centos branch,
not the contents of the PR. This makes testing PRs unreliable.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-08-03 16:50:14 +02:00
Niels de Vos
1fb77b0626 ci: correctly indent parameters for mini-e2e job
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-08-03 16:50:14 +02:00
Niels de Vos
1de6b0bba2 ci: pass Kubernetes version as parameter to mini-e2e job
Move the mini-e2e job into a template-job and generate two jobs out of
it: mini-e2e/k8s-1.17.8 and mini-e2e/k8s-1.18.5

By passing the k8s_version as variable to the job-template, and placing
it in the parameters for the mini-e2e.groovy script, all hard-coded
occurences of the Kubernetes version can be replaced by the
{k8s_version} placeholder.

See-also: https://jenkins-job-builder.readthedocs.io/en/latest/definition.html#job-template
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-08-03 14:12:25 +00:00
Niels de Vos
96dcc08223 ci: use ci/centos/mini-e2e/k8s-1.18 as context for mini-e2e job
Restarting the CI job can now be done with the following comments in a
PR:
- /test all
- /retest all
- /test ci/centos/mini-e2e
- /retest ci/centos/mini-e2e
- /test ci/centos/mini-e2e/k8s-1.18
- /retest ci/centos/mini-e2e/k8s-1.18

This is added as preparation for enabling running the mini-e2e job with
different Kubernetes versions.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-08-03 10:13:46 +00:00
Niels de Vos
f5cba3aaa8 ci: add mini-e2e Jenkins job
The new mini-e2e jobs does the following:
 - reserve a bare-metal machine
 - checkout the git repository with the PR
 - build used artifacts (container image and e2e.test executable)
 - deploy k8s and Rook in a minikube VM
 - run the e2e tests

With-contributions-from: Yug <yuggupta27@gmail.com>
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-08-03 08:11:17 +00:00
Niels de Vos
a089d8eaa1 ci: use correct refspec for fetching PRs in jjb-validate job
Also use https, instead of http for the repository.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-05-29 09:51:16 +00:00
Niels de Vos
06ef71971d ci: re-enable jjb-validate job for PRs
With the jjb/session label on the Pod, tracking the job works again.
Therefor this test can be re-enabled.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-05-29 09:51:16 +00:00
Niels de Vos
69b16771ae ci: do not automatically run jjb-validate on PRs
The jjb-validate job is not correct, it does not always return the
results of the PR.

Prevent triggering the job automatically, and only have it run when
someone leaves the '/test ci/centos/jjb-validate' comment.

Updates: #1111
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-05-28 07:36:06 +00:00
Niels de Vos
d2905081b3 ci: checkout the PR when running jjb-validate
Using the 'git' function in groovy does not allow checking out a
non-branch, so use the 'checkout' function instead.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-05-27 18:21:59 +02:00
Niels de Vos
8f4a556dcc ci: add properties for jobs/jjb-deploy.yaml
The 'scm' section also needs to be a list. Jenkins Jobs Builder does not
detect this while validating, but it cause an error while updating the
jobs in Jenkins.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-05-27 16:22:51 +02:00