Commit Graph

231 Commits

Author SHA1 Message Date
Madhu Rajanna
df047ddaaf ci: fix commitlint problem
Still seeing the issue of the commitlint
as below

fatal: unsafe repository
('/go/src/github.com/ceph/ceph-csi'
is owned by someone else)
To add an exception for this directory,
call:

git config --global --add safe.directory \
/go/src/github.com/ceph/ceph-csi
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-05-09 05:57:14 +00:00
Niels de Vos
4891e534d3 ci: prevent ERR trap inheritance for kubectl_retry
`bash -E` causes inheritance of the ERR trap into shell functions,
command substitutions, and commands executed in a subshell environment.
Because the `kubectl_retry` function depends on detection an error of a
subshell, the ERR trap is not needed to be executed. The trap contains
extra logging, and exits the script in the `rook.sh` case. The aborting
of the script is not wanted when a retry is expected to be done.

While checking for known failures, the `grep` command may exit with 1,
if there are no matches. That means, the `ret` variable will be set to
0, but there will also be an error exit status. This causes `bash -E` to
abort the function, and call the ERR trap.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-05-04 12:27:59 +00:00
Niels de Vos
9940ec38e3 ci: show all logs from kubectl
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-04-26 20:55:18 +00:00
Niels de Vos
89e2ff39f1 ci: do not count Warning: matches in kubectl_retry()
Depending on the Kubernetes version, the following warning is reported
regulary:

> Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+,
> unavailable in v1.25+

The warning is written to stderr, so skipping AlreadyExists or NotFound
is not sufficient to trigger a retry. Ignoring '^Warning:' in the stderr
output should prevent unneeded failures while deploying Rook or other
components.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-04-26 20:55:18 +00:00
Niels de Vos
1e21972956 ci: improve logging for kubectl_retry helper
Rook deployments fail quite regulary in the CI environment now. It is
not clear what the cause is, hopefully a little better logging will
guide us to the issue.

Now executing `kubectl` in a sub-shell, ensuring that the redirection of
the command lands in the right files.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-04-26 20:55:18 +00:00
Humble Chirammal
0dfc15121c deploy: change the image registry for sidecars
This commit change the image registry URL for sidecars in the
deployment from `k8s.gcr.io` to `registry.k8s.io` as
the migration is happening from former to the latter. This commit
also correct the e2e readme for the change.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-04-20 10:05:13 +00:00
Humble Chirammal
a44a97f3c5 deploy: snapshot deployment is no longer beta
Considering snapshot controllers have been moved to GA since
kube version 1.20, we no longer need to have a mention of beta
version of the same in our deployment.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-04-11 12:15:48 +00:00
Humble Chirammal
16abbbc846 build: remove cache while building container image
Reduce size of the container image by removing the cache in deploy
and devel container.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-03-28 06:09:27 +00:00
Robert Vasek
8fb5739f21 build: more flexible handling of go build tags; added ceph_preview
ceph_preview tag is needed to make new go-ceph's RADOS read/write ops available

Signed-off-by: Robert Vasek <robert.vasek@cern.ch>
2022-03-27 19:24:26 +00:00
Humble Chirammal
afeae03069 ci: fetch sidecar versions from build.env and use it
The sidecar images in minikube deployment will be fetched from
build.env and used/validated accordingly.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-03-24 14:09:13 +00:00
Yati Padia
732ab13096 cleanup: adds short description to disable goerr113 linter
This commit adds a short description with reason to
disable the goerr113 linter.

issue: #1227

Signed-off-by: Yati Padia <ypadia@redhat.com>
2022-03-16 16:18:52 +00:00
Niels de Vos
cbec296543 build: disable removed Apache Arrow repository
The CentOS 8 repository for Apache Arrow has been removed. This causes
container-image builds fail with the following error:

    Errors during downloading metadata for repository 'apache-arrow-centos':
      - Status code: 404 for https://apache.jfrog.io/artifactory/arrow/centos/8/x86_64/repodata/repomd.xml (IP: 54.190.66.70)
    Error: Failed to download metadata for repo 'apache-arrow-centos': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried

The Ceph base image has `arrow/centos/8` configured, maybe Apache Arrow
offers a CentOS Stream 8 repository now? Once the Ceph container-image
has been updated, the repository can be enabled again.

Ceph-CSI does not depend on Apache Arrow, so there is no functional
change by disabling the repository.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-02-04 10:23:58 +00:00
Humble Chirammal
de2489ed7d deploy: update csi-snapshotter sidecar to v5.0.1
This release of snapshotter has a breaking change as mentioned
in the release note:

Refer#
[1]: https://github.com/kubernetes-csi/external-snapshotter/releases/tag/v5.0.0

RBAC rules are also updated with this commit.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-02-03 19:01:57 +00:00
Madhu Rajanna
8a479d3865 deploy: use osd as failureDomain for ec pool
in Rook 1.8 templates the failureDomain is
set to host for ec pools, as we are using
single node minikube cluster, setting the
failureDomain for osd.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-01-27 04:18:01 +00:00
Madhu Rajanna
0d6db19252 deploy: disable startupProbe
As startup probe is failing on the ceph pods,
disabling startupProbe in ceph cluster CRD.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-01-27 04:18:01 +00:00
Madhu Rajanna
e5100eacd4 deploy: add check for device-health-metrics pool
Rook creates a detault pool with name
device_health_metrics for
device-health-metrics CephBlockPool CR.

device-health-metrics is added to cluster-test.yaml
in Rook 1.8

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-01-27 04:18:01 +00:00
Madhu Rajanna
493db5c183 deploy: log complete ouput of rook operator
kubectl log with labels will log only
last 10 lines by default adding tail=-1
to log complete output.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-01-27 04:18:01 +00:00
Madhu Rajanna
04f2261e60 deploy: support rook deployment for v1.8+
In Rook v1.8+ the path for the deployment articafts
are changed from `"cluster/examples/kubernetes/ceph`
to `deploy/examples`.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-01-27 04:18:01 +00:00
Madhu Rajanna
c32bf71846 deploy: re-alignment of code in rook.sh
currently, rook.sh code is not aligned properly
unside the functions, this commit does
the code alignment.

PS: this is done by vscode for me :)

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-01-27 04:18:01 +00:00
Madhu Rajanna
15ed9709d4 ci: skip new checks in golangci-lint
skipping new checks in golangci-lint as
its much work to addresses new check errors
in this PR.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-01-24 05:25:11 +00:00
Madhu Rajanna
6f0087e870 build: use cephcsi image version from build.env
we dont need to specify the image version
separately in minikube script, use the image
version defined in the build.env

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-01-20 17:53:30 +00:00
Humble Chirammal
233ac8f936 deploy: adjust minikube deployment for RWOP support
The ReadWriteOncePod feature gate need to be enabled only when we
are operating on kube 1.22 or above cluster. This commit adds the
logic to parse the kubernetes cluster at time of minikube deployment
and if it is above v1.22, enable the RWOP feature gate

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-01-11 19:40:22 +00:00
Madhu Rajanna
f3ed883df9 ci: use install.sh from golangci-lint repo
The golangci-lint install script at goreleaser.com is deprecated. Docs
now advise to install from a github link:

goreleaser/godownloader#207

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-12-22 15:47:22 +00:00
Humble Chirammal
c4eaf6e747 ci: place the markdown linter rule to restrict the line length at 80
This try to enforce the markdown rule for the line length to 80

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-12-21 14:53:13 +00:00
Madhu Rajanna
e743e06748 ci: install arch specific go in Dockerfile.test
Instead of installing the amd64 on all the
platforms, install architecture specific go
version in test dockerfile.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-12-15 05:13:36 +00:00
Madhu Rajanna
8a3fe53e87 ci: install arch specific go in Dockerfile.devel
Instead of installing the amd64 on all the
platforms, install architecture specific go
version for devel dockerfile

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-12-15 05:13:36 +00:00
Madhu Rajanna
9595c89855 ci: install findutils in dockerfile.test
currently getting find command not found
and xargs command not found when we run
the dockerfile.test. installing findutils
to fix it.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-12-13 07:32:54 +00:00
Madhu Rajanna
b114a8fdad ci: install go version specified in build.env
installed the go version specified in build.env
in the test docker image.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-12-13 07:32:54 +00:00
Madhu Rajanna
8bae51db12 ci: ignore go.sum from codespell check
as go-sum is auto generated, ignore checking
codespell on this file.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-12-13 07:32:54 +00:00
Yug Gupta
f4c87cdc22 deploy: update help command in rook.sh
Update help command in rook.sh to showcase
functionality to deploy ec-pools

Signed-off-by: Yug Gupta <yuggupta27@gmail.com>
2021-11-26 06:06:40 +00:00
Yug Gupta
3243afade5 deploy: add support to deploy ec-pool
add support to deploy ec-pool in minikube
script

Signed-off-by: Yug Gupta <yuggupta27@gmail.com>
2021-11-26 06:06:40 +00:00
Humble Chirammal
aa600754c1 deploy: remove expandCSIVolumes feature gate from deployment
The `expandCSIVolumes` feature gate is beta since kubernetes 1.16
version and we no longer wanted to explictly enable it in the
deployment.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-11-22 06:08:23 +00:00
Yug Gupta
c339d43272 deploy: deploy erasure coded pool
deploy erasure coded pool during rook
deployment to allow usage and testing
in erasure coded pools.

Signed-off-by: Yug Gupta <yuggupta27@gmail.com>
2021-11-19 14:03:21 +00:00
Madhu Rajanna
46c40fe5ad ci: skip shell check in vendor directory
skip shell check in vendor directories.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-11-16 12:03:36 +00:00
Madhu Rajanna
ec34fdd505 ci: skip codespell for retest vendor
as there are lot of spell check failures
on the vendor directory, skipping it.

skipping retest action folder as
PullRequests word is not getting ignored

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-11-16 12:03:36 +00:00
Rakshith R
191b603974 ci: remove gh action gosec linter,since it is already part of golangci
This commit removes gosec standalone linter and related parts,
since golangci linter runs gosec linter too.

Signed-off-by: Rakshith R <rar@redhat.com>
2021-11-16 12:29:56 +01:00
Rakshith R
f9c369918c ci: disable rook deployed csi drivers to speed up e2e
Signed-off-by: Rakshith R <rar@redhat.com>
2021-10-15 11:15:51 +00:00
Niels de Vos
97525f5e74 ci: add make go-test-api to GitHub Action
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-10-05 11:26:50 +00:00
Madhu Rajanna
2a8eb95966 ci: disable fsGroupChangePolicy change test for cephfs
disabling fsGroupChangePolicy test suite for
cephfs.

updates: #2017

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-09-22 10:34:41 +00:00
Madhu Rajanna
981835a073 deploy: remove extra permission for snapshot controller
removed extra PSP permissions added for the
snapshot controller deployment

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-09-22 07:12:34 +00:00
Rakshith R
6bb4ec6715 ci: set NUM_DISKS default value to 1 in minikube.sh
Signed-off-by: Rakshith R <rar@redhat.com>
2021-09-15 07:20:50 +00:00
Madhu Rajanna
3aab8dad00 ci: start minikube with psp addon
we need to start minikube with the
PodSecurityPolicy admission controller
and the pod-security-policy addon enabled
for psp.

Ref:- https://minikube.sigs.k8s.io/docs/
tutorials/using_psp/#tutorial

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-09-14 11:27:29 +00:00
Madhu Rajanna
1e037a56c1 ci: remove minikube workaround for psp
as we are no longer using older (<1.11.1)
version of minikube, removing the
workaround to support the psp with
older minikube versions.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-09-14 11:27:29 +00:00
Rakshith R
8f75a24cfd ci: use 0 as default NUM_DISKS in minikube.sh
This is done to prevent conflicts with current ci setup externally
attaching disks.

Signed-off-by: Rakshith R <rar@redhat.com>
2021-09-09 05:18:36 +00:00
Rakshith R
42a6c3c006 ci: pass $DISK_CONFIG properly to minikube start
Having double quotes around $DISK_CONFIG led to these args
not being properly passed to minikube start. This commit fixes it.

Signed-off-by: Rakshith R <rar@redhat.com>
2021-09-09 05:18:36 +00:00
Rakshith R
1b64a0a505 ci: add support to create extra disks through minikube
Signed-off-by: Rakshith R <rar@redhat.com>
2021-09-07 07:10:59 +00:00
Madhu Rajanna
cfc070b169 deploy: remove ceph-conf cm in helm installation
removed the ceph-conf configmap as a workaround
to avoid helm installation issue.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-09-03 14:14:43 +00:00
Rakshith R
dc8479f5ad ci: make csi sidecar image version configurable in minikube.sh
This commit makes csi sidecar image version configurable in
minikube.sh.

Signed-off-by: Rakshith R <rar@redhat.com>
2021-08-16 11:17:09 +00:00
Yati Padia
e077c1fdf5 cleanup: run codespell on containerized testing
This commit adds a new target codespell to the
make containerized-test.

Fixes: #2229

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-08-12 09:42:54 +05:30
Rakshith R
7fba62dd47 ci: internally create & delete cephcsi namespace in install-helm.sh
This ensures the kubectl call is retried with kubectl_retry function.

Updates: #2309

Signed-off-by: Rakshith R <rar@redhat.com>
2021-08-11 08:42:21 +00:00