Commit Graph

33 Commits

Author SHA1 Message Date
Niels de Vos
6a7e6c841f ci: pre-pull rook/ceph image from local registry
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-19 12:48:32 +00:00
Niels de Vos
b9cffc1b42 ci: pass registry to podman helper functions
Functions with Groovy can not use `def ci_registry` as the variable is
not in the scope. Pass the registry to the podman_login() and
podman_pull() functions instead.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-18 12:31:22 +00:00
Niels de Vos
ea5985fa3a ci: fix calling podman_login()
A typo when calling podman_log() causes CI jobs to fail.

Fixes: 1eec379 "ci: pre-pull Ceph base-image and cephcsi:devel for mini-e2e-helm jobs"
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-18 12:02:25 +00:00
Niels de Vos
f36ef72a19 ci: pre-pull nginx and vault images from local registry
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-18 11:49:10 +00:00
Niels de Vos
dd10e66a98 ci: move podman2minikube() into its own script
This way, it can easier be re-used for other container images.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-18 11:49:10 +00:00
Niels de Vos
42bfd19da6 ci: introduce and use podman helper functions
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-18 11:49:10 +00:00
Niels de Vos
9887a08377 ci: the image in the CI registry is called "ceph-csi"
The image is named after the project, so "ceph-csi".

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-17 13:21:50 +00:00
Niels de Vos
e9523b3ace ci: store podman-login results in ~/.podman-auth.json
It seems that "podman pull" does not consume the authentication details
from ~/.docker/config.json, so store the results of "podman login" in
~/.podman-auth.json and use the file for all "podman pull" commands.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-17 13:00:36 +00:00
Niels de Vos
6b4dffbd79 ci: store results of podman-login in ~/.docker/config.json
Once the ssh command finishes, the runtime directory is removed and the
results of "podman login" are lost. By storing the results in the
standard Docker configuration file, subsequent "podman pull" commands
will be able to re-use the authentication details.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-17 12:37:27 +00:00
Niels de Vos
baac40c136 ci: escape command to be executed over ssh to fetch BASE_IMAGE
The ${BASE_IMAGE} variable gets expanded by running the ssh command.
This becomes an empty variable, so the "echo" part of the command does
not output anything.

By escaping the command, there is no variable substitution, and the
BASE_IMAGE variable should get stored in the variable.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-17 11:46:59 +00:00
Niels de Vos
ea281737fe ci: ssh into the bare-metal machine
Missed the CICO_NODE to ssh into, that has now been added too.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-17 11:20:32 +01:00
Niels de Vos
3c82db25e3 ci: correctly fetch BASE_IMAGE from build.env
The BASE_IMAGE variable was not stored in the variable so that the CI
job can consume it. By using sh(), this should be the case now.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-17 10:12:32 +00:00
Niels de Vos
b5123e6134 ci: use podman to login on the CI registry
The CI registry contains several container images that can be pulled
instead of using Docker Hub.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-17 09:51:16 +00:00
Niels de Vos
a8a6ef27a6 ci: use "podman login --username" instead of "--user"
It seems the --user option does not exist, but is called --username.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-17 09:19:37 +00:00
Niels de Vos
d7172453ca ci: pull cached images from local registry
This reduces the number of images that get pulled from Docker Hub where
pull limits affect the jobs.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-17 08:50:03 +00:00
Niels de Vos
466ada3860 ci: use github/refs/pull/<pr-id>/merge instead of doing a rebase
When fetching refs/pull/<pr-id/merge from GitHub, there is no need to do
a manual rebase. This makes things easier, as a the scripted rebases
sometimes cause CI jobs to fail.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-28 04:12:48 +00:00
Niels de Vos
51fa5cca48 ci: use refs/pull/<id>/merge to prevent need for rebases
refs/pull/<id>/head might not contain the most current state of the
branch. In case other PRs got merged, the PR under test needs rebasing.
GitHub offers refs/pull/<id>/merge to checkout the rebased PR, use that
in the CI jobs.

In case refs/pull/<id>/merge is not available, it means the PR can not
be rebased on its target branch. This will cause the CI job to fail, but
GitHub also will have a message about rebase conflicts.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-21 13:29:06 +00:00
Niels de Vos
126fcd4cb5 ci: move skip-e2e-check after GitHub API usage
When the [ci/skip/e2e] label is set on PRs, the withCredentials()
statement is aborted, but the other stages still continue. This causes
the tests to run, which is not what we want when the label is added.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-20 06:37:25 +00:00
Niels de Vos
ed671ec207 ci: use the credentials-binding plugin for the GitHub API Token
The standard credentials() function does not seem to do what we need. So
use the credentials-binding Jenkins plugin instead.

See-also: https://www.jenkins.io/doc/pipeline/steps/credentials-binding/
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-19 11:28:12 +00:00
Niels de Vos
a6ca8e31dc ci: get the GitHub API Token before starting a node
It still seems that the environment is not set when the GitHub API is
called. Maybe things work better when the environment is set before
starting the cico-workspace node.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-19 09:41:35 +00:00
Niels de Vos
89783afcf5 ci: set GITHUB_API_TOKEN in the environment for all stages
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-15 07:43:53 +00:00
Niels de Vos
2fb4e9fd22 ci: use environment{} to fetch the GitHub API token
The `credentials()` function might only work in the `environment` block
in the Pipelines. At the moment, running the 'skip ci/skip/e2e label'
stage always reports 'Error: 401 Client Error: Unauthorized'.

Fixes: e0d49908 (ci: fetch GITHUB_API_TOKEN from Jenkins credential store)
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-13 15:26:41 +00:00
Niels de Vos
e0d49908f8 ci: fetch GITHUB_API_TOKEN from Jenkins credential store
Fetch the named credential "github-api-token" from the Jenkins
configuration. This is a "personal access token" that has been created
with the ceph-csi-bot user account.

CC: @ceph-csi-bot
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-13 12:05:57 +00:00
Niels de Vos
e353d406eb ci: pass major k8s version to jobs, detect patch release
Jobs can now pass the wanted Kubernetes major version (like '1.19') to
the Jenkins Pipeline scripts. The Pipelines detect the most recent patch
release for the major version with the new get_patch_release.py script.

This causes the CI Job status context to not have the patch number (last
digit of the release) included anymore. Restarting a test will only need
the major version number, as does updating the Mergify configuration.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-09-15 17:02:04 +05:30
Niels de Vos
8806d8e69b ci: skip mini-e2e jobs if PR has the 'ci/skip/e2e' label
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-08-24 08:51:46 +00:00
Yug
79adaef3d1 ci: Increase timeout for mini-e2e job
Due to a strict timeout, the job
tends to abort sometimes. Increasing the
timeout to allow sufficient time for
tests to execute.

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-08-18 06:32:52 +00:00
Niels de Vos
0e19f37f61 ci: skip mini-e2e for doc-only PRs
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-08-12 16:04:26 +00:00
Yug
4f4513fa0a ci: Correct the stage name
Curently the stage name directly
prints the name of the variable
in place of substituting it.
This is a fix for that issue.

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-08-04 04:02:18 +00:00
Yug
a71785d0f2 ci: use double quotes for variables
Using double quotes as variables are
expanded inside them.
The script fails currently as it is
unable to expand the variables.

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-08-03 17:42:15 +00:00
Niels de Vos
1de6b0bba2 ci: pass Kubernetes version as parameter to mini-e2e job
Move the mini-e2e job into a template-job and generate two jobs out of
it: mini-e2e/k8s-1.17.8 and mini-e2e/k8s-1.18.5

By passing the k8s_version as variable to the job-template, and placing
it in the parameters for the mini-e2e.groovy script, all hard-coded
occurences of the Kubernetes version can be replaced by the
{k8s_version} placeholder.

See-also: https://jenkins-job-builder.readthedocs.io/en/latest/definition.html#job-template
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-08-03 14:12:25 +00:00
Niels de Vos
8fbeefa989 ci: use ceph-csi:ci/centos for fetching job scripts
Commit f5cba3aaa8 added the mini-e2e job, but still referred to the
temporary location that was used for testing the job. As everything is
available in the ceph-csi:ci/centos repository:branch, there is no need
to refer to the temporary location.

Reported-by: Yug <yuggupta27@gmail.com>
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-08-03 11:22:13 +00:00
Niels de Vos
d8f098fb47 ci: disable debug sleep on failure of mini-e2e
While debugging issues with the job itself, a sleep has been very
useful. PRs that have been rebased on the master branch contain all the
deployment fixes that are needed for the job to pass. There is no need
anymore to run into the long sleep when the job fails.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-08-03 10:32:11 +00:00
Niels de Vos
f5cba3aaa8 ci: add mini-e2e Jenkins job
The new mini-e2e jobs does the following:
 - reserve a bare-metal machine
 - checkout the git repository with the PR
 - build used artifacts (container image and e2e.test executable)
 - deploy k8s and Rook in a minikube VM
 - run the e2e tests

With-contributions-from: Yug <yuggupta27@gmail.com>
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-08-03 08:11:17 +00:00