Commit Graph

255 Commits

Author SHA1 Message Date
Niels de Vos
b2e31dbe1e cleanup: no need for disabling the apache-arrow-centor repository
The apache-arrow-centor repository is not available in current Ceph
container-images, there is no need to try to disable the repository
anymore.

See-also: https://github.com/ceph/ceph-container/pull/1990
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2023-06-02 11:08:20 +00:00
Niels de Vos
bc167cc6f4 ci: install Helm with script located on GitHub
Installing Helm fails often in the CI. The Helm documentation does not
point to `https://git.io/get_helm.sh` anymore, but to a location on
GitHub. To make it easier to update the location in the future, it has
now been added to `build.env`, just like the `HELM_VERSION`.

See-also: https://helm.sh/docs/intro/install/
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2023-05-22 13:27:12 +00:00
riya-singhal31
44612fe34c ci: fix shell check failures
Signed-off-by: riya-singhal31 <rsinghal@redhat.com>
2023-04-20 15:24:35 +00:00
riya-singhal31
f12cd9c986 ci: fix codespell failure
Signed-off-by: riya-singhal31 <rsinghal@redhat.com>
2023-04-20 15:24:35 +00:00
Lennart Jern
e6c97b9593 cleanup: Fix paths and syntax
Minor cleanup of changed paths and deprecated syntax

Signed-off-by: Lennart Jern <lennart.jern@est.tech>
2023-01-10 08:00:39 +00:00
riya-singhal31
539686329f ci: fix mdl related failures
This commit address the issue-
https://github.com/ceph/ceph-csi/issues/3448.

Signed-off-by: riya-singhal31 <rsinghal@redhat.com>
2022-11-17 08:25:10 +00:00
Niels de Vos
93d32c49c8 build: fix ShellCheck issue in scripts/test-go.sh
With the updated Fedora 37 container-image, a new version of ShellCheck
gets installed. This version is a little more strict and complains about
the array expansion in `scripts/test-go.sh`.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-11-16 08:01:46 +00:00
Niels de Vos
774beef838 ci: install openssl for Fedora 37 testing image
GitHub Workflows fail installing Helm if the `openssl` package is not
available. Fedora 36 installs `openssl` by default, Fedora 37 does not.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-11-16 08:01:46 +00:00
Humble Chirammal
1695240f54 build: remove travis CI scripts for functest and helm
considering we have moved out of travis and replaced github
action, this commit remove the functional test and helmtest
scripts.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-11-09 13:12:57 +00:00
BOSSER, Bastien
dea07aa184 deploy: add commonLabels value
Signed-off-by: BOSSER, Bastien <bastien.bosser@atos.net>
2022-11-02 11:28:18 +00:00
Madhu Rajanna
0e294d66e2 ci: remove podsecurity feature-gate
remove the podsecurity feature-gate
from minikube.sh, because of it
kubernetes 1.25.0 deployment is failing

fixes: #3358

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-10-26 14:32:51 +00:00
Rakshith R
b3837d44ce ci: fix mdl configuration
This commit makes the following changes:
`Please replace \":code_blocks => false\" \`
`by \":ignore_code_blocks => true\" in your configuration.`
Some rules are ignore for the time being,
these will be fixed later on.

Signed-off-by: Rakshith R <rar@redhat.com>
2022-10-18 07:47:33 +00:00
Marcel Lauhoff
f89076b1d7 scripts: Add env to set minikube iso url
Make iso url configurable to use pre-release minikube images or
local-built (file://)

Signed-off-by: Marcel Lauhoff <marcel.lauhoff@suse.com>
2022-10-17 17:33:52 +00:00
Niels de Vos
fa97875dc9 ci: fail installing Helm if wget is unavailable
In case `wget` is not installed, downloading the Helm release will fail.
The `install-helm.sh` script won't return a fatal error in that case,
and CI jobs continue running in an environment that is not ready.

By adding a check that exist the script with a failure, the CI will now
correctly report a problem when Helm can not be downloaded.

See-also: #3430
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-10-13 10:45:35 +00:00
Niels de Vos
0cba72485c ci: add support for VM_DRIVER=podman to scripts/minikube.sh
When running on AWE EC2 virtual-machines, we'll use Podman instead of
installing a VM. The "none" driver might work as well, but it requires
additional dependencies to be installed, which may change over time with
new minikube or Kubernetes releases. Hopefully the Podman driver is less
affected with changes in dependencies.

Depends-on: #3419
Closes: #3415
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-10-13 10:45:35 +00:00
Madhu Rajanna
96a3aabe5a deploy: remove psp from cephcsi
as PSP is deprecated in kubernetes 1.21
and will be removed in kubernetes 1.25
removing the existing PSP related templates
from the repo and updated the required documents.

fixes #1988

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-08-23 07:53:46 +00:00
Madhu Rajanna
5c9f63bada ci: disable nosnakecase static check
All the CSI spec imports are having snakecase
we cannot fix it, instead of disabling all imports
skipping the nosnakecase static check.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-08-10 17:51:26 +00:00
Humble Chirammal
c9773db3f3 ci: remove check for snapshot controller installation and cleanup
At present, the check is performed to validate the version of kube
is v1.17 and this commit remove the same.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-08-05 12:02:45 +00:00
Humble Chirammal
5aabd4e1d2 deploy: remove the snapshot controller installation check
no need to have 1.17 kube version check anymore  before we install
snapshot controller.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-08-05 12:02:45 +00:00
Niels de Vos
04889e66db ci: verify that Ceph Mgr is running
The Ceph v17.2.2 container-image fails to start Ceph Mgr. This causes
issues while the e2e test suite is running. It is better to check if
Ceph Mgr is available, before continuing with the rest of the CI job.

Updates: #3259
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-07-26 12:47:51 +00:00
Madhu Rajanna
f7a9558728 ci: add check for .mgr pool in rook.sh
Rook v1.9.x creates pool with name .mgr
for builtin-mgr CephBlockPool CR.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-06-16 04:09:42 +00:00
Madhu Rajanna
9e5cad173d rebase: skip newly added complicated check in golangci-lint
skipping maintidx,exhaustruct and containedctx tests
which seems complicated to fix.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-06-03 12:55:54 +00:00
Humble Chirammal
66631cdbf7 deploy: enable RecoverVolumeExpansionFailure featuregate
this commit enable RecoverVolumeExpansionFailure featuregate in
minikube deployment.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-05-25 18:03:16 +00:00
Madhu Rajanna
92272d41c5 ci: disable PodSecurity featuregate
PodSecurity featuregate is beta in kubernetes
1.23 and its causing problem for the existing
tests. This PR disables the PodSecurity featuregate
for now and will be enabled later.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-05-09 09:16:12 +00:00
Madhu Rajanna
df047ddaaf ci: fix commitlint problem
Still seeing the issue of the commitlint
as below

fatal: unsafe repository
('/go/src/github.com/ceph/ceph-csi'
is owned by someone else)
To add an exception for this directory,
call:

git config --global --add safe.directory \
/go/src/github.com/ceph/ceph-csi
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-05-09 05:57:14 +00:00
Niels de Vos
4891e534d3 ci: prevent ERR trap inheritance for kubectl_retry
`bash -E` causes inheritance of the ERR trap into shell functions,
command substitutions, and commands executed in a subshell environment.
Because the `kubectl_retry` function depends on detection an error of a
subshell, the ERR trap is not needed to be executed. The trap contains
extra logging, and exits the script in the `rook.sh` case. The aborting
of the script is not wanted when a retry is expected to be done.

While checking for known failures, the `grep` command may exit with 1,
if there are no matches. That means, the `ret` variable will be set to
0, but there will also be an error exit status. This causes `bash -E` to
abort the function, and call the ERR trap.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-05-04 12:27:59 +00:00
Niels de Vos
9940ec38e3 ci: show all logs from kubectl
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-04-26 20:55:18 +00:00
Niels de Vos
89e2ff39f1 ci: do not count Warning: matches in kubectl_retry()
Depending on the Kubernetes version, the following warning is reported
regulary:

> Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+,
> unavailable in v1.25+

The warning is written to stderr, so skipping AlreadyExists or NotFound
is not sufficient to trigger a retry. Ignoring '^Warning:' in the stderr
output should prevent unneeded failures while deploying Rook or other
components.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-04-26 20:55:18 +00:00
Niels de Vos
1e21972956 ci: improve logging for kubectl_retry helper
Rook deployments fail quite regulary in the CI environment now. It is
not clear what the cause is, hopefully a little better logging will
guide us to the issue.

Now executing `kubectl` in a sub-shell, ensuring that the redirection of
the command lands in the right files.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-04-26 20:55:18 +00:00
Humble Chirammal
0dfc15121c deploy: change the image registry for sidecars
This commit change the image registry URL for sidecars in the
deployment from `k8s.gcr.io` to `registry.k8s.io` as
the migration is happening from former to the latter. This commit
also correct the e2e readme for the change.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-04-20 10:05:13 +00:00
Humble Chirammal
a44a97f3c5 deploy: snapshot deployment is no longer beta
Considering snapshot controllers have been moved to GA since
kube version 1.20, we no longer need to have a mention of beta
version of the same in our deployment.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-04-11 12:15:48 +00:00
Humble Chirammal
16abbbc846 build: remove cache while building container image
Reduce size of the container image by removing the cache in deploy
and devel container.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-03-28 06:09:27 +00:00
Robert Vasek
8fb5739f21 build: more flexible handling of go build tags; added ceph_preview
ceph_preview tag is needed to make new go-ceph's RADOS read/write ops available

Signed-off-by: Robert Vasek <robert.vasek@cern.ch>
2022-03-27 19:24:26 +00:00
Humble Chirammal
afeae03069 ci: fetch sidecar versions from build.env and use it
The sidecar images in minikube deployment will be fetched from
build.env and used/validated accordingly.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-03-24 14:09:13 +00:00
Yati Padia
732ab13096 cleanup: adds short description to disable goerr113 linter
This commit adds a short description with reason to
disable the goerr113 linter.

issue: #1227

Signed-off-by: Yati Padia <ypadia@redhat.com>
2022-03-16 16:18:52 +00:00
Niels de Vos
cbec296543 build: disable removed Apache Arrow repository
The CentOS 8 repository for Apache Arrow has been removed. This causes
container-image builds fail with the following error:

    Errors during downloading metadata for repository 'apache-arrow-centos':
      - Status code: 404 for https://apache.jfrog.io/artifactory/arrow/centos/8/x86_64/repodata/repomd.xml (IP: 54.190.66.70)
    Error: Failed to download metadata for repo 'apache-arrow-centos': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried

The Ceph base image has `arrow/centos/8` configured, maybe Apache Arrow
offers a CentOS Stream 8 repository now? Once the Ceph container-image
has been updated, the repository can be enabled again.

Ceph-CSI does not depend on Apache Arrow, so there is no functional
change by disabling the repository.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-02-04 10:23:58 +00:00
Humble Chirammal
de2489ed7d deploy: update csi-snapshotter sidecar to v5.0.1
This release of snapshotter has a breaking change as mentioned
in the release note:

Refer#
[1]: https://github.com/kubernetes-csi/external-snapshotter/releases/tag/v5.0.0

RBAC rules are also updated with this commit.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-02-03 19:01:57 +00:00
Madhu Rajanna
8a479d3865 deploy: use osd as failureDomain for ec pool
in Rook 1.8 templates the failureDomain is
set to host for ec pools, as we are using
single node minikube cluster, setting the
failureDomain for osd.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-01-27 04:18:01 +00:00
Madhu Rajanna
0d6db19252 deploy: disable startupProbe
As startup probe is failing on the ceph pods,
disabling startupProbe in ceph cluster CRD.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-01-27 04:18:01 +00:00
Madhu Rajanna
e5100eacd4 deploy: add check for device-health-metrics pool
Rook creates a detault pool with name
device_health_metrics for
device-health-metrics CephBlockPool CR.

device-health-metrics is added to cluster-test.yaml
in Rook 1.8

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-01-27 04:18:01 +00:00
Madhu Rajanna
493db5c183 deploy: log complete ouput of rook operator
kubectl log with labels will log only
last 10 lines by default adding tail=-1
to log complete output.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-01-27 04:18:01 +00:00
Madhu Rajanna
04f2261e60 deploy: support rook deployment for v1.8+
In Rook v1.8+ the path for the deployment articafts
are changed from `"cluster/examples/kubernetes/ceph`
to `deploy/examples`.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-01-27 04:18:01 +00:00
Madhu Rajanna
c32bf71846 deploy: re-alignment of code in rook.sh
currently, rook.sh code is not aligned properly
unside the functions, this commit does
the code alignment.

PS: this is done by vscode for me :)

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-01-27 04:18:01 +00:00
Madhu Rajanna
15ed9709d4 ci: skip new checks in golangci-lint
skipping new checks in golangci-lint as
its much work to addresses new check errors
in this PR.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-01-24 05:25:11 +00:00
Madhu Rajanna
6f0087e870 build: use cephcsi image version from build.env
we dont need to specify the image version
separately in minikube script, use the image
version defined in the build.env

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-01-20 17:53:30 +00:00
Humble Chirammal
233ac8f936 deploy: adjust minikube deployment for RWOP support
The ReadWriteOncePod feature gate need to be enabled only when we
are operating on kube 1.22 or above cluster. This commit adds the
logic to parse the kubernetes cluster at time of minikube deployment
and if it is above v1.22, enable the RWOP feature gate

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-01-11 19:40:22 +00:00
Madhu Rajanna
f3ed883df9 ci: use install.sh from golangci-lint repo
The golangci-lint install script at goreleaser.com is deprecated. Docs
now advise to install from a github link:

goreleaser/godownloader#207

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-12-22 15:47:22 +00:00
Humble Chirammal
c4eaf6e747 ci: place the markdown linter rule to restrict the line length at 80
This try to enforce the markdown rule for the line length to 80

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-12-21 14:53:13 +00:00
Madhu Rajanna
e743e06748 ci: install arch specific go in Dockerfile.test
Instead of installing the amd64 on all the
platforms, install architecture specific go
version in test dockerfile.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-12-15 05:13:36 +00:00
Madhu Rajanna
8a3fe53e87 ci: install arch specific go in Dockerfile.devel
Instead of installing the amd64 on all the
platforms, install architecture specific go
version for devel dockerfile

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-12-15 05:13:36 +00:00