Commit Graph

23 Commits

Author SHA1 Message Date
Niels de Vos
f129719f18 ci: quote username for podman_login()
Jenkins does not like the passing of the username as variable to the
podman_login() function. Calling the function results in an error like

    Warning: A secret was passed to "sh" using Groovy String interpolation, which is insecure.
        Affected argument(s) used the following variable(s): [CREDS_USER]
        See https://jenkins.io/redirect/groovy-string-interpolation for details.
    + ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no root@n7.pufty.ci.centos.org 'podman login --authfile=~/.podman-auth.json --username=$CREDS_USER --password=**** registry-****.apps.ocp.ci.centos.org'
    Username: Error: error getting username and password: error reading username: EOF

By single quoting the username, just like the password, it may work
better.

Fixes: aca3745e2 ("ci: do not use Groovy string interpolation for credentials")
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-07-15 10:44:27 +02:00
Niels de Vos
aca3745e24 ci: do not use Groovy string interpolation for credentials
Jenkins warns in the output of CI jobs about the following:

    Warning: A secret was passed to "sh" using Groovy String interpolation, which is insecure.
        Affected argument(s) used the following variable(s): [CREDS_PASSWD, CREDS_USER]
        See https://jenkins.io/redirect/groovy-string-interpolation for details.

Variable with 'single quotes' and without the {curly brackets} are
expecred to not be affected. There is some indirection in the strings
passed to the `sh` function, so this approach might not fix it?

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-07-15 07:48:41 +00:00
Niels de Vos
c04a319aa9 ci: only abort on doc-change when running for PRs
In case a job has been started without a PR (manual, or timed), the
current checked out branch matches the original as there are not
additional changes in the tree. There is no need to abort the jobs when
the skip-doc-change.sh script did not detect any non-doc changes, as
there are no changes at all.

Updates: #1963
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-04-09 09:53:22 +00:00
Niels de Vos
8f84e592d5 ci: do not re-checkout current branch
When tests are started manually (through the Jenkins webui), there is no
PR associated with the job. That means the `git_since` and `ref` are
equal. Trying to create a new branch named `ref` will not work, as the
branch was already created when cloning the repository with `git_since`.

With this change, Jenkins jobs can be started manually. This makes it
possible to run regular/nightly jobs as well.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-04-08 08:38:11 +00:00
Niels de Vos
e6b70c494e ci: prevent parallel builds from causing conflicts
When the container image needs to be rebuild, two parallel jobs will try
to attempt that. With recent versions of Podman, this now fails.

When the image needs to be rebuild, do so in the stage where it would
otherwise get pulled. This makes sure the image gets build only once.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-03-03 05:48:42 +00:00
Niels de Vos
1c2974d49e ci: the "master" branch got renamed to "devel"
Closes: #1193
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-03-01 05:22:06 +00:00
Niels de Vos
a26772188a ci: pull BASE_IMAGE from local registry
The CI scripts pull all container images from the local CI registry. If
the image name starts with "docker.io/", the images will be pushed into
the test environment as "docker.io/docker.io/ceph/ceph:v15". This image
will not be used by the tests, so things can still fail in case Docker
Hub has reached the pull rate-limit.

By dropping the additional "docker.io/" from the BASE_IMAGE name, the
image gets pushed as "docker.io/ceph/ceph:v15" so the tests will use it
automatically.

Groovy-syntax: https://www.baeldung.com/groovy-remove-string-prefix#using-regex
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-30 08:47:18 +00:00
Niels de Vos
468b6cd67d ci: pull images from local registry directly
The mirror option of the Docker Registry container is very limited and
prevents updating or manually pushing images to the registry. Instead,
it tries to push the images to the docker.io, which is not what we need.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-26 16:26:57 +00:00
Niels de Vos
5ae8fb7c9b ci: add configuration for the proxy/mirror registry
This makes it possible to pull images from Docker Hub through the local
container image registry in the CI OpenShift deployment. The registry in
the CI is configured with the 'cephcsibot' account so that pulling
images is accounted towards the account, and not anonymous consumers
within the whole CentOS CI.

There should be no need to manually sync the images between the local
registry and Docker Hub anymore.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-24 05:09:38 +00:00
Niels de Vos
b9cffc1b42 ci: pass registry to podman helper functions
Functions with Groovy can not use `def ci_registry` as the variable is
not in the scope. Pass the registry to the podman_login() and
podman_pull() functions instead.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-18 12:31:22 +00:00
Niels de Vos
ea5985fa3a ci: fix calling podman_login()
A typo when calling podman_log() causes CI jobs to fail.

Fixes: 1eec379 "ci: pre-pull Ceph base-image and cephcsi:devel for mini-e2e-helm jobs"
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-18 12:02:25 +00:00
Niels de Vos
7796f52043 ci: pre-pull ceph base image for containerized-tests job
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-18 11:49:10 +00:00
Niels de Vos
f67ece89a9 ci: split "go mod verify" into its own stage
In order to speed up testing, "go mod verify" can be run parallel from
other tests. Downloading and verifying all dependencies is the most time
consuming step in the containerized-test CI job.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-04 05:52:01 +00:00
Niels de Vos
09c4191a6f ci: enable USE_PULLED_IMAGE to prevent image builds
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-04 05:52:01 +00:00
Niels de Vos
4ba6e51f85 ci: use ssh() helper function for containerized-tests
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-04 05:52:01 +00:00
Niels de Vos
f14bf59856 ci: use cached image for containerized-tests
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-04 05:52:01 +00:00
Niels de Vos
466ada3860 ci: use github/refs/pull/<pr-id>/merge instead of doing a rebase
When fetching refs/pull/<pr-id/merge from GitHub, there is no need to do
a manual rebase. This makes things easier, as a the scripted rebases
sometimes cause CI jobs to fail.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-28 04:12:48 +00:00
Niels de Vos
693d7b953e ci: skip containerized-tests for doc-only PRs
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-08-12 16:04:26 +00:00
Niels de Vos
954b2f42ea ci: request CentOS-8 bare-metal machines
Currently CentOS-7 machines were requested. CentOS-8 has been out for a
while now, and is stable for running manual jobs. There is nothing
preventing us from using CentOS-8 bare-metal machines for testing.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-08-01 11:24:35 +05:30
Niels de Vos
8259fe953f ci: use parameters set by github-pull-request-builder
The parameters are set in the environment. It seems the best way to
check if the parameters are available, is with `params.<arg> == null`.

See-also: https://www.jenkins.io/doc/book/pipeline/getting-started/#global-variable-reference#
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-05-25 17:10:06 +02:00
Yug Gupta
748ef17e42 ci: retry if no machine is immediately available
To prevent the failure of job due to unavailability
of a machine immediately, retry mechanism is used.

If unable to reserve a machine, it will retry every
5 mins for 30 times to avoid job failure.

Signed-off-by: Yug Gupta <ygupta@redhat.com>
2020-05-18 16:14:16 +02:00
Yug Gupta
f6229ae237 ci: execute build and test stages in parallel
Execution of build and test stages in parallel
will reduce the job execution time.

Signed-off-by: Yug Gupta <ygupta@redhat.com>
2020-05-18 14:01:05 +02:00
Yug Gupta
45cd8c311a ci: Add a stage to run containerized-test
Running containerized test prior to the build
stage to ensure that build starts only after
tests pass.

Signed-off-by: Yug Gupta <ygupta@redhat.com>
2020-05-15 17:01:26 +02:00