jenkins-job-builder does not seem to work with urllib3 2.x, force using
version 1.x as a workaround.
Updates: #3946
Signed-off-by: Niels de Vos <ndevos@ibm.com>
CentOS Stream 8 does not provide a module named 'dataclasses'. It seems
current versions of Jenkins Jobs Builder require that. Python 3.7
offers the module by default, so using a new base distribution with
Python 3.9 has the module too.
See-also: #3909
Signed-off-by: Niels de Vos <ndevos@ibm.com>
In the old OpenShift cluster the registry consumer 800+ GB of data. Once
running the garbage collection manually, the consumption reduced to a
little over 8GB. Let's be nice users of the infrastructure and run
garbage collection weekly.
Signed-off-by: Niels de Vos <ndevos@ibm.com>
ShellCheck failed with the following warning:
```
SC2086 (info): Double quote to prevent globbing and word splitting.
Did you mean:
git config --global --add safe.directory "${PWD}"
```
Signed-off-by: Niels de Vos <ndevos@ibm.com>
When Jenkins Jobs have been modified, they should get deployed in the
Jenkins environment. This seems to fail with the following error:
```
Initialized empty Git repository in /opt/build/.git/
fatal: detected dubious ownership in repository at '/opt/build'
To add an exception for this directory, call:
git config --global --add safe.directory /opt/build
```
By marking /opt/build as a safe directory in the global git
configuration file, this problem should not occur anymore.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
The container images for Jenkins Job Builder and mirroring images were
still using the EOL CentOS 8 base image. Rebuilding those
container-images fails as the CentOS mirrors do not provide the
repositories for installing package anymore.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
registry.centos.org is not officially
maintained by the CentOS infrastructure
team. The container images on quay.io are
the official once and we should use
those instead.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
This was updated to prevent the following error.
Using non-groupfied API resources is deprecated and will be removed in a
future release, update apiVersion to "template.openshift.io/v1" for your
resource.
Signed-off-by: Rakshith R <rar@redhat.com>
Instead of using a mirror, the CI registry is now pupulated with
container images that get pulled and tagged as if they get from Docker
Hub or other locations.
This is more of a manual mirror, as the Docker Registry mirror
functionality is not flexible enough for our usecase (push images
without providing them on docker.io).
Signed-off-by: Niels de Vos <ndevos@redhat.com>
This makes it possible to pull images from Docker Hub through the local
container image registry in the CI OpenShift deployment. The registry in
the CI is configured with the 'cephcsibot' account so that pulling
images is accounted towards the account, and not anonymous consumers
within the whole CentOS CI.
There should be no need to manually sync the images between the local
registry and Docker Hub anymore.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
In case the '--cmd' option is missing, usage() will be called and the
script exits with 0. A missing option is a failure, so make usage()
return and handle the exit status when usage() got called instead.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
There is no need to re-use different layers with the images that are
built. The separation of the layers causes more time to be used while
pulling the images, so flattening makes speeds up the CI jobs a little
(about 30 seconds from 2+ minutes).
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Add the ability to run JJB from private branch/fork
jjb-deploy.yaml:
- Add GIT_REPO parameter (not required) with default value
jjb.sh:
- Changing PS4 for more indicative debug prompt
- Adding flags to get the repo and branch
jjb-deploy.yaml:
- Calling the jjb.sh with the correct flags
Signed-off-by: liranmauda <liran.mauda@gmail.com>
The BuildConfig can be imported in the OpenShift environment so that
Ceph-CSI images can be built automatically. These images will be cached
in a private container image registry, which can speed up CI jobs that
currently rebuild the images every time.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
With a private container image registry, the images that CI jobs use can
be cached and re-used. This speeds up the CI jobs, as building the
container images takes up the majority of the runtime for some jobs.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Old jobs stay behind (and active) when jobs get updated with new
versions. This mostly affects the mini-e2e jobs at the moment.
There is no need to keep old job around, so delete them while deploying
updates to jobs.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
GIT_REF can point to a ref that it not part of a branch and can not
easily be checked out (like a GitHub PR). Checking out FETCH_HEAD after
fetching the GIT_REF works.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
The batch job templates for OpenShift use a GIT_REF variable that is
used by the jobs to checkout the repository. This defaults to
"ci/centos", but it never is adjusted when jjb-validate runs against a
PR.
With the new GIT_REF environment variable in the jjb.sh script, the
variable can now be used to checkout a PR from gitHub and run the
validation against that.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
On the new OCP4 cluster, deleting the batch job results in deletion of
the pod as well. Deleting the pod when it does not exist anymore results
in a failure. Ignoring the potential failure makes the job pass again.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Currently the number of Pods is growing in the OpenShift Console. The
pods are all Completed, so not running anymore, but there is also no
need to keep them around.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
By using a template, it becomes possible to identify the Pod that has
been started by the Batch Job.
This prevents the script from getting the logs from an incorrect (old)
container.
Fixes: #1111
Signed-off-by: Niels de Vos <ndevos@redhat.com>
The directory that gets constructed can be like
`/opt/build/deploy/..//jobs` and this causes jenkins-jobs to fail.
Enternig the WORKDIR and passing the local ./jobs directory to
jenkins-jobs makes things clearer.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
There is a new .commitlintrc.yml file in the root of the repository.
This causes jenkins-job-builder to complains that the file is not
confirm the Jenkins Job Builder format/specification.
jenkins_jobs.errors.JenkinsJobsException: The topmost collection in file '/opt/build/.commitlintrc.yml' must be a list, not a <class 'collections.OrderedDict'>
Move all Jenkins Job Builder .yaml files to jobs/ so that all the
Jenkins job configurations are kept together without other files.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
16:7 error wrong indentation: expected 8 but found 6 (indentation)
19:9 error wrong indentation: expected 10 but found 8 (indentation)
Signed-off-by: Niels de Vos <ndevos@redhat.com>
./deploy/jjb-deploy.yaml
16:7 error wrong indentation: expected 8 but found 6 (indentation)
19:9 error wrong indentation: expected 10 but found 8 (indentation)
26:9 error wrong indentation: expected 10 but found 8 (indentation)
30:7 error wrong indentation: expected 8 but found 6 (indentation)
Signed-off-by: Niels de Vos <ndevos@redhat.com>
In ./deploy/jjb.sh line 11:
oc get pod/${1} --no-headers -o=jsonpath='{.status.phase}'
^--^ SC2086: Double quote to prevent globbing and word splitting.
In ./deploy/jjb.sh line 29:
cd $(dirname ${0})
^----------------^ SC2164: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.
^-------------^ SC2046: Quote this to prevent word splitting.
^--^ SC2086: Double quote to prevent globbing and word splitting.
In ./deploy/jjb.sh line 31:
oc create -f jjb-${CMD}.yaml
^----^ SC2086: Double quote to prevent globbing and word splitting.
In ./deploy/jjb.sh line 36:
jjb_pod=$(oc get pods --no-headers -l job-name=jjb-${CMD} -o=jsonpath='{.items[0].metadata.name}')
^----^ SC2086: Double quote to prevent globbing and word splitting.
In ./deploy/jjb.sh line 37:
[ ${?} = 0 ] && [ -n "${jjb_pod}" ] && break
^--^ SC2181: Check exit code directly with e.g. 'if mycmd;', not indirectly with $?.
In ./deploy/jjb.sh line 46:
[ ${?} = 0 ] && ( [ "${status}" = "Succeeded" ] || [ "${status}" = "Failed" ] ) && break
^--^ SC2181: Check exit code directly with e.g. 'if mycmd;', not indirectly with $?.
^-- SC2235: Use { ..; } instead of (..) to avoid subshell overhead.
In ./deploy/jjb.sh line 54:
oc delete --wait -f jjb-${CMD}.yaml
^----^ SC2086: Double quote to prevent globbing and word splitting.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
These files make it possible to
- build a container image that includes Jenkins Job Builder
- store the container image in the OpenShift ImageStream
- use the container to validate or deploy the Jenkins Jobs
More details about the files are included in the README.md.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
