Commit Graph

243 Commits

Author SHA1 Message Date
Marcel Lauhoff
f89076b1d7 scripts: Add env to set minikube iso url
Make iso url configurable to use pre-release minikube images or
local-built (file://)

Signed-off-by: Marcel Lauhoff <marcel.lauhoff@suse.com>
2022-10-17 17:33:52 +00:00
Niels de Vos
fa97875dc9 ci: fail installing Helm if wget is unavailable
In case `wget` is not installed, downloading the Helm release will fail.
The `install-helm.sh` script won't return a fatal error in that case,
and CI jobs continue running in an environment that is not ready.

By adding a check that exist the script with a failure, the CI will now
correctly report a problem when Helm can not be downloaded.

See-also: #3430
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-10-13 10:45:35 +00:00
Niels de Vos
0cba72485c ci: add support for VM_DRIVER=podman to scripts/minikube.sh
When running on AWE EC2 virtual-machines, we'll use Podman instead of
installing a VM. The "none" driver might work as well, but it requires
additional dependencies to be installed, which may change over time with
new minikube or Kubernetes releases. Hopefully the Podman driver is less
affected with changes in dependencies.

Depends-on: #3419
Closes: #3415
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-10-13 10:45:35 +00:00
Madhu Rajanna
96a3aabe5a deploy: remove psp from cephcsi
as PSP is deprecated in kubernetes 1.21
and will be removed in kubernetes 1.25
removing the existing PSP related templates
from the repo and updated the required documents.

fixes #1988

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-08-23 07:53:46 +00:00
Madhu Rajanna
5c9f63bada ci: disable nosnakecase static check
All the CSI spec imports are having snakecase
we cannot fix it, instead of disabling all imports
skipping the nosnakecase static check.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-08-10 17:51:26 +00:00
Humble Chirammal
c9773db3f3 ci: remove check for snapshot controller installation and cleanup
At present, the check is performed to validate the version of kube
is v1.17 and this commit remove the same.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-08-05 12:02:45 +00:00
Humble Chirammal
5aabd4e1d2 deploy: remove the snapshot controller installation check
no need to have 1.17 kube version check anymore  before we install
snapshot controller.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-08-05 12:02:45 +00:00
Niels de Vos
04889e66db ci: verify that Ceph Mgr is running
The Ceph v17.2.2 container-image fails to start Ceph Mgr. This causes
issues while the e2e test suite is running. It is better to check if
Ceph Mgr is available, before continuing with the rest of the CI job.

Updates: #3259
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-07-26 12:47:51 +00:00
Madhu Rajanna
f7a9558728 ci: add check for .mgr pool in rook.sh
Rook v1.9.x creates pool with name .mgr
for builtin-mgr CephBlockPool CR.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-06-16 04:09:42 +00:00
Madhu Rajanna
9e5cad173d rebase: skip newly added complicated check in golangci-lint
skipping maintidx,exhaustruct and containedctx tests
which seems complicated to fix.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-06-03 12:55:54 +00:00
Humble Chirammal
66631cdbf7 deploy: enable RecoverVolumeExpansionFailure featuregate
this commit enable RecoverVolumeExpansionFailure featuregate in
minikube deployment.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-05-25 18:03:16 +00:00
Madhu Rajanna
92272d41c5 ci: disable PodSecurity featuregate
PodSecurity featuregate is beta in kubernetes
1.23 and its causing problem for the existing
tests. This PR disables the PodSecurity featuregate
for now and will be enabled later.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-05-09 09:16:12 +00:00
Madhu Rajanna
df047ddaaf ci: fix commitlint problem
Still seeing the issue of the commitlint
as below

fatal: unsafe repository
('/go/src/github.com/ceph/ceph-csi'
is owned by someone else)
To add an exception for this directory,
call:

git config --global --add safe.directory \
/go/src/github.com/ceph/ceph-csi
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-05-09 05:57:14 +00:00
Niels de Vos
4891e534d3 ci: prevent ERR trap inheritance for kubectl_retry
`bash -E` causes inheritance of the ERR trap into shell functions,
command substitutions, and commands executed in a subshell environment.
Because the `kubectl_retry` function depends on detection an error of a
subshell, the ERR trap is not needed to be executed. The trap contains
extra logging, and exits the script in the `rook.sh` case. The aborting
of the script is not wanted when a retry is expected to be done.

While checking for known failures, the `grep` command may exit with 1,
if there are no matches. That means, the `ret` variable will be set to
0, but there will also be an error exit status. This causes `bash -E` to
abort the function, and call the ERR trap.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-05-04 12:27:59 +00:00
Niels de Vos
9940ec38e3 ci: show all logs from kubectl
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-04-26 20:55:18 +00:00
Niels de Vos
89e2ff39f1 ci: do not count Warning: matches in kubectl_retry()
Depending on the Kubernetes version, the following warning is reported
regulary:

> Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+,
> unavailable in v1.25+

The warning is written to stderr, so skipping AlreadyExists or NotFound
is not sufficient to trigger a retry. Ignoring '^Warning:' in the stderr
output should prevent unneeded failures while deploying Rook or other
components.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-04-26 20:55:18 +00:00
Niels de Vos
1e21972956 ci: improve logging for kubectl_retry helper
Rook deployments fail quite regulary in the CI environment now. It is
not clear what the cause is, hopefully a little better logging will
guide us to the issue.

Now executing `kubectl` in a sub-shell, ensuring that the redirection of
the command lands in the right files.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-04-26 20:55:18 +00:00
Humble Chirammal
0dfc15121c deploy: change the image registry for sidecars
This commit change the image registry URL for sidecars in the
deployment from `k8s.gcr.io` to `registry.k8s.io` as
the migration is happening from former to the latter. This commit
also correct the e2e readme for the change.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-04-20 10:05:13 +00:00
Humble Chirammal
a44a97f3c5 deploy: snapshot deployment is no longer beta
Considering snapshot controllers have been moved to GA since
kube version 1.20, we no longer need to have a mention of beta
version of the same in our deployment.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-04-11 12:15:48 +00:00
Humble Chirammal
16abbbc846 build: remove cache while building container image
Reduce size of the container image by removing the cache in deploy
and devel container.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-03-28 06:09:27 +00:00
Robert Vasek
8fb5739f21 build: more flexible handling of go build tags; added ceph_preview
ceph_preview tag is needed to make new go-ceph's RADOS read/write ops available

Signed-off-by: Robert Vasek <robert.vasek@cern.ch>
2022-03-27 19:24:26 +00:00
Humble Chirammal
afeae03069 ci: fetch sidecar versions from build.env and use it
The sidecar images in minikube deployment will be fetched from
build.env and used/validated accordingly.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-03-24 14:09:13 +00:00
Yati Padia
732ab13096 cleanup: adds short description to disable goerr113 linter
This commit adds a short description with reason to
disable the goerr113 linter.

issue: #1227

Signed-off-by: Yati Padia <ypadia@redhat.com>
2022-03-16 16:18:52 +00:00
Niels de Vos
cbec296543 build: disable removed Apache Arrow repository
The CentOS 8 repository for Apache Arrow has been removed. This causes
container-image builds fail with the following error:

    Errors during downloading metadata for repository 'apache-arrow-centos':
      - Status code: 404 for https://apache.jfrog.io/artifactory/arrow/centos/8/x86_64/repodata/repomd.xml (IP: 54.190.66.70)
    Error: Failed to download metadata for repo 'apache-arrow-centos': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried

The Ceph base image has `arrow/centos/8` configured, maybe Apache Arrow
offers a CentOS Stream 8 repository now? Once the Ceph container-image
has been updated, the repository can be enabled again.

Ceph-CSI does not depend on Apache Arrow, so there is no functional
change by disabling the repository.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-02-04 10:23:58 +00:00
Humble Chirammal
de2489ed7d deploy: update csi-snapshotter sidecar to v5.0.1
This release of snapshotter has a breaking change as mentioned
in the release note:

Refer#
[1]: https://github.com/kubernetes-csi/external-snapshotter/releases/tag/v5.0.0

RBAC rules are also updated with this commit.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-02-03 19:01:57 +00:00
Madhu Rajanna
8a479d3865 deploy: use osd as failureDomain for ec pool
in Rook 1.8 templates the failureDomain is
set to host for ec pools, as we are using
single node minikube cluster, setting the
failureDomain for osd.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-01-27 04:18:01 +00:00
Madhu Rajanna
0d6db19252 deploy: disable startupProbe
As startup probe is failing on the ceph pods,
disabling startupProbe in ceph cluster CRD.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-01-27 04:18:01 +00:00
Madhu Rajanna
e5100eacd4 deploy: add check for device-health-metrics pool
Rook creates a detault pool with name
device_health_metrics for
device-health-metrics CephBlockPool CR.

device-health-metrics is added to cluster-test.yaml
in Rook 1.8

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-01-27 04:18:01 +00:00
Madhu Rajanna
493db5c183 deploy: log complete ouput of rook operator
kubectl log with labels will log only
last 10 lines by default adding tail=-1
to log complete output.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-01-27 04:18:01 +00:00
Madhu Rajanna
04f2261e60 deploy: support rook deployment for v1.8+
In Rook v1.8+ the path for the deployment articafts
are changed from `"cluster/examples/kubernetes/ceph`
to `deploy/examples`.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-01-27 04:18:01 +00:00
Madhu Rajanna
c32bf71846 deploy: re-alignment of code in rook.sh
currently, rook.sh code is not aligned properly
unside the functions, this commit does
the code alignment.

PS: this is done by vscode for me :)

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-01-27 04:18:01 +00:00
Madhu Rajanna
15ed9709d4 ci: skip new checks in golangci-lint
skipping new checks in golangci-lint as
its much work to addresses new check errors
in this PR.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-01-24 05:25:11 +00:00
Madhu Rajanna
6f0087e870 build: use cephcsi image version from build.env
we dont need to specify the image version
separately in minikube script, use the image
version defined in the build.env

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-01-20 17:53:30 +00:00
Humble Chirammal
233ac8f936 deploy: adjust minikube deployment for RWOP support
The ReadWriteOncePod feature gate need to be enabled only when we
are operating on kube 1.22 or above cluster. This commit adds the
logic to parse the kubernetes cluster at time of minikube deployment
and if it is above v1.22, enable the RWOP feature gate

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-01-11 19:40:22 +00:00
Madhu Rajanna
f3ed883df9 ci: use install.sh from golangci-lint repo
The golangci-lint install script at goreleaser.com is deprecated. Docs
now advise to install from a github link:

goreleaser/godownloader#207

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-12-22 15:47:22 +00:00
Humble Chirammal
c4eaf6e747 ci: place the markdown linter rule to restrict the line length at 80
This try to enforce the markdown rule for the line length to 80

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-12-21 14:53:13 +00:00
Madhu Rajanna
e743e06748 ci: install arch specific go in Dockerfile.test
Instead of installing the amd64 on all the
platforms, install architecture specific go
version in test dockerfile.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-12-15 05:13:36 +00:00
Madhu Rajanna
8a3fe53e87 ci: install arch specific go in Dockerfile.devel
Instead of installing the amd64 on all the
platforms, install architecture specific go
version for devel dockerfile

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-12-15 05:13:36 +00:00
Madhu Rajanna
9595c89855 ci: install findutils in dockerfile.test
currently getting find command not found
and xargs command not found when we run
the dockerfile.test. installing findutils
to fix it.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-12-13 07:32:54 +00:00
Madhu Rajanna
b114a8fdad ci: install go version specified in build.env
installed the go version specified in build.env
in the test docker image.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-12-13 07:32:54 +00:00
Madhu Rajanna
8bae51db12 ci: ignore go.sum from codespell check
as go-sum is auto generated, ignore checking
codespell on this file.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-12-13 07:32:54 +00:00
Yug Gupta
f4c87cdc22 deploy: update help command in rook.sh
Update help command in rook.sh to showcase
functionality to deploy ec-pools

Signed-off-by: Yug Gupta <yuggupta27@gmail.com>
2021-11-26 06:06:40 +00:00
Yug Gupta
3243afade5 deploy: add support to deploy ec-pool
add support to deploy ec-pool in minikube
script

Signed-off-by: Yug Gupta <yuggupta27@gmail.com>
2021-11-26 06:06:40 +00:00
Humble Chirammal
aa600754c1 deploy: remove expandCSIVolumes feature gate from deployment
The `expandCSIVolumes` feature gate is beta since kubernetes 1.16
version and we no longer wanted to explictly enable it in the
deployment.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-11-22 06:08:23 +00:00
Yug Gupta
c339d43272 deploy: deploy erasure coded pool
deploy erasure coded pool during rook
deployment to allow usage and testing
in erasure coded pools.

Signed-off-by: Yug Gupta <yuggupta27@gmail.com>
2021-11-19 14:03:21 +00:00
Madhu Rajanna
46c40fe5ad ci: skip shell check in vendor directory
skip shell check in vendor directories.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-11-16 12:03:36 +00:00
Madhu Rajanna
ec34fdd505 ci: skip codespell for retest vendor
as there are lot of spell check failures
on the vendor directory, skipping it.

skipping retest action folder as
PullRequests word is not getting ignored

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-11-16 12:03:36 +00:00
Rakshith R
191b603974 ci: remove gh action gosec linter,since it is already part of golangci
This commit removes gosec standalone linter and related parts,
since golangci linter runs gosec linter too.

Signed-off-by: Rakshith R <rar@redhat.com>
2021-11-16 12:29:56 +01:00
Rakshith R
f9c369918c ci: disable rook deployed csi drivers to speed up e2e
Signed-off-by: Rakshith R <rar@redhat.com>
2021-10-15 11:15:51 +00:00
Niels de Vos
97525f5e74 ci: add make go-test-api to GitHub Action
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-10-05 11:26:50 +00:00