The run-k8s-external-storage-e2e.sh script downloads the kubernetes-test
components, creates a configuration for RBD and CephFS drivers and runs
the external-storage tests on the current cluster.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
The jjb-validate job creates a Batch Job in OCP. This job has a fixed
name, and fails to get created when one with the same name already
exists. There is no need to run jjb-validate concurrently, the number of
changes for CI jobs are not blocked by this serialization.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
refs/pull/<id>/head might not contain the most current state of the
branch. In case other PRs got merged, the PR under test needs rebasing.
GitHub offers refs/pull/<id>/merge to checkout the rebased PR, use that
in the CI jobs.
In case refs/pull/<id>/merge is not available, it means the PR can not
be rebased on its target branch. This will cause the CI job to fail, but
GitHub also will have a message about rebase conflicts.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
When the [ci/skip/e2e] label is set on PRs, the withCredentials()
statement is aborted, but the other stages still continue. This causes
the tests to run, which is not what we want when the label is added.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
It still seems that the environment is not set when the GitHub API is
called. Maybe things work better when the environment is set before
starting the cico-workspace node.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
The latest Kubernetes patch release (v1.18.10) can not be deployed with
minikube. Selecting version 1.18.9 until the problems with minikube are
addressed.
Updates: #1588
Signed-off-by: Niels de Vos <ndevos@redhat.com>
The latest Kubernetes patch release (v1.19.3) can not be deployed with
minikube. Selecting version 1.19.2 until the problems with minikube are
addressed.
Updates: #1588
Signed-off-by: Niels de Vos <ndevos@redhat.com>
In case the version that is passed with --version=... contains the patch
release already, the latest patch release will not be detected, but the
passed version will be returned (enables forcing a particular version in
CI jobs).
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Anonymous users have a low (currently 60) rate limit of API calls.
Authorized users have a much higher (currently 5000) limit. Print the
available, used and remainling rate limit when the GitHub API returns a
403 messages like:
Error: 403 Client Error: rate limit exceeded
In case of anonymous access to the API, and hitting the limit, the
message in the logs will also include the following:
Rate limit (limit/used/remaining): 60/60/0
Signed-off-by: Niels de Vos <ndevos@redhat.com>
The `credentials()` function might only work in the `environment` block
in the Pipelines. At the moment, running the 'skip ci/skip/e2e label'
stage always reports 'Error: 401 Client Error: Unauthorized'.
Fixes: e0d49908 (ci: fetch GITHUB_API_TOKEN from Jenkins credential store)
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Fetch the named credential "github-api-token" from the Jenkins
configuration. This is a "personal access token" that has been created
with the ceph-csi-bot user account.
CC: @ceph-csi-bot
Signed-off-by: Niels de Vos <ndevos@redhat.com>
These script now check if GITHUB_API_TOKEN is set in the environment,
and use that for authenticating. There is no username needed when an API
token is used, but it may not be empty, so it is set to "unused".
Signed-off-by: Niels de Vos <ndevos@redhat.com>
On occasion the scripts that use the GitHub API fail without logging any
useful error:
./scripts/get_github_labels.py --id=1568 --has-label=ci/skip/e2e
Traceback (most recent call last):
File "./scripts/get_github_labels.py", line 71, in <module>
main()
File "./scripts/get_github_labels.py", line 55, in main
names = get_names(json)
File "./scripts/get_github_labels.py", line 40, in get_names
names.append(label['name'])
TypeError: string indices must be integers
While debugging, it seems that the limit of API calls is reached:
403 Client Error: rate limit exceeded
It is useful to have failure messages reported in the output of the CI
jobs for future potential troubleshooting.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Old jobs stay behind (and active) when jobs get updated with new
versions. This mostly affects the mini-e2e jobs at the moment.
There is no need to keep old job around, so delete them while deploying
updates to jobs.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Rebase current checked out PR on top of parent branch.
This will then prevent commitlint to check complete
history of 'ci/centos' branch.
Signed-off-by: Yug <yuggupta27@gmail.com>
Installing Docker CE on CentOS-8 currently conflicts with Podman. Both
tools can not be installed at the same time.
Instead of using the Docker command to push the image to to minikube VM,
read the image from stdin over ssh and load it with the Docker command
that is available inside the VM.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Instead of accessing the upgrade version from environment
variable, set the variable directly in configuration file.
Signed-off-by: Yug <yuggupta27@gmail.com>
Since UPGRADE_VERSION is not a groovy variable, adding an
escape character is required for proper execution of
upgrade-testing job.
Signed-off-by: Yug <yuggupta27@gmail.com>
GIT_REF can point to a ref that it not part of a branch and can not
easily be checked out (like a GitHub PR). Checking out FETCH_HEAD after
fetching the GIT_REF works.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
The batch job templates for OpenShift use a GIT_REF variable that is
used by the jobs to checkout the repository. This defaults to
"ci/centos", but it never is adjusted when jjb-validate runs against a
PR.
With the new GIT_REF environment variable in the jjb.sh script, the
variable can now be used to checkout a PR from gitHub and run the
validation against that.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
The GitHub Pull Requests Builder plugin sets the ghprbPullId parameter.
There is no need to make it configurable, and it even overloads the
value from the plugin by the default value.
This cauesed the jjb-validate job to run against the ci/centos branch
for each PR, not against the contents of the PR itself.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
The Jenkins Jobs Builder files reference 'sandbox: true' as an option,
but this does not exist (anymore). Having the option causes warnings in
the Jenkins WebUI for admin users.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Jobs can now pass the wanted Kubernetes major version (like '1.19') to
the Jenkins Pipeline scripts. The Pipelines detect the most recent patch
release for the major version with the new get_patch_release.py script.
This causes the CI Job status context to not have the patch number (last
digit of the release) included anymore. Restarting a test will only need
the major version number, as does updating the Mergify configuration.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
By using this script, we can pass Kubernetes versions like v1.19 to the
CI jobs. The jobs are then responsible for detecting the a full release
version, including the '.patch' suffix.
Versions are passed as 1.19, releases have the 'v' prefix as in v1.19.1.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
removed 1.17 as we are testing only two versions
in centos CI. as v1.19 is released, we will be
testing v1.18 and v1.19 in CI.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
By passing `--id=<id>` the labels of the given Pull-Request or Issue get
printed. When adding `--has-label=<label>`, the script verifies if the
label is set and returns 0 on succes, or 1 on failure.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
if the commit header is not present in the list of
the component we maintain. the commitlint check should
show error instead of showing the warning.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit 52243d0140)
The --history does not take an argument, similar to --help. Move it out
of ARGUMENT_LIST so 'getopt' does not complain about it.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
After fetching the target branch for the PR, the GIT_SINCE ref in the
git repository may not be set (in CI environments). This causes 'git
rebase' to fail. Use FETCH_HEAD instead, so that the checked out PR gets
rebased correctly.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
In case the history of the branch is needed (commitlint job), add an
option --history to skip cloning the repository with --depth=1.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
By passing REBASE=1 to the commitlint make target, the current checked
out PR will be rebased on top of the parent branch. This should result
in a linear history that commitlint can parse correctly.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
When Mergify adds a merge commit to the branch that is being tested with
commitlint, the tool tries to detect the most recent changes based on
the newly merged commit. This is for most PRs the master branch, and
that contains incorrect commit messages in the history. Because of this,
commitlint will fail.
By adding an option (REBASE=1) to the commitlint make target, CI jobs
can request a rebase so that the history of the PR becomes linear again
and commitlint should be able to detect only the new commits.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
