Commit Graph

311 Commits

Author SHA1 Message Date
Madhu Rajanna
b22fe5f059 ci: remove kubernetes 1.20 from CI
removing kubernetes test run with 1.20
as with  #2705 we dont need 1.20 CI
results to merge a PR and we have kube 1.23
latest release.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-12-13 04:46:06 +00:00
Madhu Rajanna
31d037eb95 ci: run tests on kubernetes 1.23 by default
The e2e tests are running fine on kubernetes
1.23. This commit runs the tests on kubernetes
1.23 by default.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-12-10 06:58:46 +00:00
Madhu Rajanna
d351e00992 ci: run tests on kubernetes 1.23
This commits removes the older version (1.19
and 1.20) from testing and adds the kuberenetes
1.23 to run on request for now.

Note:- Once the kubernetes 1.23 job is stable
will run it by default

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-12-09 09:06:18 +00:00
Yug Gupta
904a352257 ci: deploy erasure coded pool
deploy erasure coded pool while deploying
rook ceph cluster in the CI.

Signed-off-by: Yug Gupta <yuggupta27@gmail.com>
2021-11-25 09:54:22 +00:00
Rakshith R
8cf5619da3 ci: use refs/pull/<id>/merge in k8s external storage tests too
This change was missed in
https://github.com/ceph/ceph-csi/pull/1617.

Signed-off-by: Rakshith R <rar@redhat.com>
2021-11-22 09:13:09 +00:00
Madhu Rajanna
92c8f723f9 ci: use official CentOS container location
registry.centos.org is not officially
maintained by the CentOS infrastructure
team. The container images on quay.io are
the official once and we should use
those instead.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-11-19 07:51:31 +00:00
Madhu Rajanna
a07fb3bcfc ci: update images for vault 1.8.5
updating ci scripts to pull the
vault images from ci registry.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-11-19 07:12:54 +00:00
Niels de Vos
e122cff477 e2e: do not delete the namespace and contents after a failure
When a failure occurs, by default the test namespace is removed. This
makes it impossible to fetch the logs of the containers where the
failure was discovered. Pass --delete-namespace-on-failure=false as an
additional argument to the `run-e2e` make target, so that the namespace
is kept.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-10-18 14:48:23 +00:00
Niels de Vos
83e483bb8d ci: automatically run k8s-e2e-external-storage/1.22 job
The job is currently stable and passes usually. It can now be used for
validation.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-09-29 07:13:16 +00:00
Rakshith R
b2300b0835 ci: use internal minikube extra disks functionality
Minikube from v1.23.0 supports adding extra disks in
kvm2 and hyperkit VM drivers. This commit makes use of
that functionality and removes steps to attack disks
externally.

Signed-off-by: Rakshith R <rar@redhat.com>
2021-09-14 05:28:42 +00:00
Niels de Vos
ca4ec105c2 ci: use pre-pulled image for building e2e.test
The 'mini-e2e' job uses the pre-pulled container image for building the
e2e.test executable. The building of the executable takes less than 2
minutes, while building with preparing the container image takes close
to 5 minutes.

This reduces consumption of network bandwith, and might speed up other
parallel builds.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-08-19 09:33:35 +00:00
Prasanna Kumar Kalever
a373882099 ci: dump the rbd-nbd logs
print the rbd-nbd logs from the VM location /var/log/ceph/

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-08-18 15:20:32 +00:00
Rakshith R
2f24c7f47d ci: use csi sidecar image versions from build.env if available
This commit makes changes necessary to use csi sidecar image versions
from build.env if available.

PR https://github.com/ceph/ceph-csi/pull/2217 adds these
variables to `build.env`.

Signed-off-by: Rakshith R <rar@redhat.com>
2021-08-16 11:17:26 +00:00
Yug Gupta
2be0306c82 ci: run tests on k8s v1.22 in place of v1.19
As kubernetes v1.19 is heading towards its EOL
on 2021-09-30, run tests on kubernetes v1.22
and require it to pass for merging.

Signed-off-by: Yug Gupta <yuggupta27@gmail.com>
2021-08-12 11:32:43 +00:00
Rakshith R
935b261be1 ci: remove create & delete namespace commands from helm jobs
Since this will be done internally by intall_helm.sh script
with kubectl_retry.

Fixes: #2309
Depends-on: #2377

Signed-off-by: Rakshith R <rar@redhat.com>
2021-08-12 04:17:29 +00:00
Rakshith R
70c6414e1e ci: add --ignore-not-found=true to delete namespace in helm cleanup
This commit is added to facilitate merging of
https://github.com/ceph/ceph-csi/pull/2377 into devel and release
branches.

https://github.com/ceph/ceph-csi/pull/2378 will remove create &
delete namespace cmds from helm e2e jobs later on.

Signed-off-by: Rakshith R <rar@redhat.com>
2021-08-11 06:10:46 +00:00
Rakshith R
e6c04e1414 ci: set ttlSecondsAfterFinished=10 to ensure job status can be checked
Signed-off-by: Rakshith R <rar@redhat.com>
2021-08-09 14:47:18 +00:00
Rakshith R
6255ec9a7a ci: update Template yamls apiVersion to template.openshift.io/v1
This was updated to prevent the following error.
Using non-groupfied API resources is deprecated and will be removed in a
future release, update apiVersion to "template.openshift.io/v1" for your
resource.

Signed-off-by: Rakshith R <rar@redhat.com>
2021-08-09 14:47:18 +00:00
Niels de Vos
c90023ad5d ci: add the root user to the libvirt group
When starting a minikube VM, there are always warning messages like
this:

    X libvirt group membership check failed:
    user is not a member of the appropriate libvirt group

The CI jobs run as root, so minikube works just fine. By adding the root
user to the libvirt group, the warning is hopefully removed.

Note that the libvirt group may not exist yet, as the packages will get
installed in a later stage. This change also adds a check to create the
libvirt group in case it is missing.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-07-16 09:59:45 +00:00
Niels de Vos
f129719f18 ci: quote username for podman_login()
Jenkins does not like the passing of the username as variable to the
podman_login() function. Calling the function results in an error like

    Warning: A secret was passed to "sh" using Groovy String interpolation, which is insecure.
        Affected argument(s) used the following variable(s): [CREDS_USER]
        See https://jenkins.io/redirect/groovy-string-interpolation for details.
    + ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no root@n7.pufty.ci.centos.org 'podman login --authfile=~/.podman-auth.json --username=$CREDS_USER --password=**** registry-****.apps.ocp.ci.centos.org'
    Username: Error: error getting username and password: error reading username: EOF

By single quoting the username, just like the password, it may work
better.

Fixes: aca3745e2 ("ci: do not use Groovy string interpolation for credentials")
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-07-15 10:44:27 +02:00
Niels de Vos
aca3745e24 ci: do not use Groovy string interpolation for credentials
Jenkins warns in the output of CI jobs about the following:

    Warning: A secret was passed to "sh" using Groovy String interpolation, which is insecure.
        Affected argument(s) used the following variable(s): [CREDS_PASSWD, CREDS_USER]
        See https://jenkins.io/redirect/groovy-string-interpolation for details.

Variable with 'single quotes' and without the {curly brackets} are
expecred to not be affected. There is some indirection in the strings
passed to the `sh` function, so this approach might not fix it?

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-07-15 07:48:41 +00:00
Yug
b4fcd2b585 ci: fix helm job failure
Due to the quotes, the job is accepting
both --deploy-sc and --deploy-secret as
as single parameter.
Removing quotes to help it consider them
as diferent.

Signed-off-by: Yug <yuggupta27@gmail.com>
2021-07-06 09:08:09 +00:00
Yug
b8c039528d ci: avoid deployment of sc and secret on v3.3
For release 3.3, we will not be deploying storageclass
and secret on helm installation on ci.
Moving forward, devel and all the future release will
have the deployment enabled by default in the ci.

Signed-off-by: Yug <yuggupta27@gmail.com>
2021-07-06 07:44:47 +00:00
Yug
f69921afb1 ci: remove temp job mini-e2e-helm-sc
Since the job passes, we can now get rid
of the temp job.

Signed-off-by: Yug <yuggupta27@gmail.com>
2021-07-06 07:44:47 +00:00
Yug
b9a1698a89 e2e: deploy storageclass and secret
deploy secret and storageclass with
deployment via helm.

Signed-off-by: Yug <yuggupta27@gmail.com>
2021-07-06 07:44:47 +00:00
Yug
4a102b44cb e2e: force deploy storageclass and secret
Add a temp job to test the deployment
of sc and secret via helm.

Signed-off-by: Yug <yuggupta27@gmail.com>
2021-06-30 15:55:53 +00:00
Niels de Vos
432ff08758 ci: skip Ephemeral-volume kubernetes external-storage tests
Ceph-CSI does not suport (inline) Ephemeral-volumes. Testing this will
continue to fail. The driver configuration can not be used to disable
testing of this feature, so it is done by skipping the tests by pattern
matching.

Updates: #2017
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-06-30 13:40:08 +00:00
Niels de Vos
2909e85e23 ci: add Kubernetes 1.22 as optional version for CI jobs
Kubernetes 1.22 is in the release process and can be used for testing
already. The CI jobs will be available and can be triggered by leaving a
comment in the PRs like

    /test ci/centos/mini-e2e-helm/k8s-1.22

See-also: https://github.com/kubernetes/sig-release/tree/master/releases/release-1.22
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-06-30 09:44:54 +00:00
Niels de Vos
a3e235774a ci: run jobs with Kubernetes 1.21
Kubernetes 1.21 is the latest stable release, and Ceph-CSI should be
tested with that. Currently 1.19 is still supported too, so we will need
to run the CI jobs with 1.19, 1.20 and 1.21.

See-also: https://kubernetes.io/releases/
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-06-30 09:44:54 +00:00
Yug
0786d15951 ci: pass --helm-test argument for helm tests
To identify if a test runs on ceph-csi deployed
via helm charts, pass --helm-test parameter with
the E2E args.

Signed-off-by: Yug <yuggupta27@gmail.com>
2021-06-24 08:13:15 +00:00
Yug
69091b9273 ci: pass namespace after --namespace flag
Since we are adding support for deployment of sc and
secret via flags, to help script recognize when an
unknown string is passed as a namespace, use
--namespace flag before entering the namespace.

Signed-off-by: Yug <yuggupta27@gmail.com>
2021-06-24 08:13:15 +00:00
Rakshith R
eb4d9259a9 ci: get status from all namespaces
This commit adds capability to get events, pods,
and logs from all namespaces.

Signed-off-by: Rakshith R <rar@redhat.com>
2021-05-20 11:35:19 +00:00
Rakshith R
6337a9fc06 ci: update upgrade-version to v3.3.1 for upgrade tests
This commit updates `upgrade-version` to `v3.3.1` for
e2e upgrade tests.

Signed-off-by: Rakshith R <rar@redhat.com>
2021-05-19 05:17:58 +00:00
Niels de Vos
0dd488374f ci: get logs from all pods, not only the 1st
The JSON formatting will return the name of the first pod, not all pods
on the list. By useing `items[*]` the names of all pods will be
returned. This causes all logs to be fetched, instead of only the logs
from the 1st pod.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-05-18 04:42:40 +00:00
Rakshith R
ad2c02bdc0 ci: get logs from all containers in pod in system-status.sh
Signed-off-by: Rakshith R <rar@redhat.com>
2021-05-17 12:08:28 +00:00
Madhu Rajanna
4a6b08256b ci: fix running ceph -s in toolbox pod
we need to pass `--` when executing a command
in the container or else it will fail.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-05-17 11:26:20 +00:00
Madhu Rajanna
0cb5abc6fb ci: log previous instance of the container in a pod
if the pod is crashed and restarted the current logs
is not helpful. Logging with `-p` might help us
to log the previous container.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-05-17 11:13:32 +00:00
Niels de Vos
a75430ce21 ci: rebuild infra container images after merge
When the `ci/centos` branch has been merged, the container images that
are built from it may need to be rebuilt. Instead of conditionally
rebuilding images only on a change, just rebuild them always, as there
are very few changes in the CI branch anyway.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-05-17 10:12:28 +00:00
Niels de Vos
06409fcbd6 ci: add CronJob for automated image mirroring in the CI registry
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-05-17 06:25:51 +00:00
Niels de Vos
2f6400fe57 ci: add mirror-images container image for automated mirroring
The ImageStream and BuildConfig use the Containerfile to build a
container image in the OpenShift registry so that automated image
mirroring can be configured.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-05-17 06:25:51 +00:00
Niels de Vos
4ec5b6f045 ci: add script and config to mirror images to local CI registry
This script reads images.txt and copies the images from docker.io (or
other registries) to the local registry in the CI.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-05-17 06:25:51 +00:00
Niels de Vos
c40a055628 ci: log events from the rook-ceph namespace on failure
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-05-14 16:45:24 +00:00
Niels de Vos
0049638e64 ci: get logs from all pods in the rook-ceph namespace on failure
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-05-14 16:45:24 +00:00
Niels de Vos
7ebc8306ef ci: get the logs from the Ceph cluster pods
It seems that `/var/log/rook` inside the VM does not contain any files.
Getting the logs from the Pods through kubectl may not be as stable, but
it should get some logs when minikube is still/again available.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-05-14 13:06:17 +00:00
Niels de Vos
410db81215 ci: include status of the Rook deployment on failure
More details of the Rook (and Ceph) deployment should be useful when
troubleshooting CI failures. This now includes the status of the most
important Kubernetes objects, and all the logs Ceph stores on the host.

Updates: #1969
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-05-12 14:42:09 +00:00
Yug
86d5c17ba4 ci: use "-T" to display the filesystem type
Provide filesystem type information with logs

Signed-off-by: Yug <yuggupta27@gmail.com>
2021-05-12 09:36:53 +00:00
Niels de Vos
bae519db07 ci: use "top -w" for untruncated wide output
Without the `-w` argument, the output of `top` gets truncated, and the
commandline of the processes is not comlete. It would be useful to eb
able to tell which command uses 100% CPU in an output like:

  17377 root      20   0  110.8m   8.2m   0.0   0.1   0:00.89 S  `- containerd+
  17414 167       20   0 1036.7m  59.6m   0.0   0.4   0:03.47 S      `- ceph-o+
  40875 root      20   0  283.9m  30.4m 100.0   0.2   0:00.23 R      `- ceph

Updates: #1969
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-05-12 05:12:09 +00:00
Niels de Vos
e599e95f25 ci: in case of a failure, return error after logging system status
It seems that it is required to re-throw the error after a catch{..}
block. Without this, and a successful execution of system-status.sh, the
CI jobs get marked as SUCCESS, even when there was a failure.

Fixes: e36155283 "ci: run system-status.sh in case a job fails"
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-05-11 13:13:39 +00:00
Niels de Vos
5b03721a58 ci: copy system-status.sh script to the bare metal node
Without the script on the node, it can not be executed...

Fixes: e36155283 "ci: run system-status.sh in case a job fails"
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-05-11 12:33:03 +00:00
Niels de Vos
e36155283b ci: run system-status.sh in case a job fails
The new `system-status.sh` script logs the status of the host and the
minikube VM. This gets executed when a CI job fails, and should aid in
troubleshooting spurious failures.

Updates: #1969
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-05-11 11:27:01 +00:00