Commit Graph

89 Commits

Author SHA1 Message Date
Humble Chirammal
5cb8f80c10 build: enable ceph_pre_quincy tag
This commit add ceph_pre_quincy tag to the build. The main
reason being the changes or new APIs exposed in go-ceph version
v0.17.0 is part of this Ceph build tag. Enabling the same for
the reason.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-08-18 11:42:57 +00:00
Prasanna Kumar Kalever
23c324898a rebase: bump go-ceph version to v0.16.0
go-ceph v0.16.0 contains subvolume metadata APIs and subvolume snapshot
metadata APIs.

Please note, as the APIs can not be tested in the go-ceph CI, it requires
build-tag `ceph_ci_untested`.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2022-06-15 13:17:09 +00:00
Niels de Vos
282c33cb58 rebase: use go-ceph version with NFS-Admin API
The NFS-Admin API has been added to go-ceph v0.15.0. As the API can not
be tested in the go-ceph CI, it requires build-tag `ceph_ci_untested`.
This additional build-tag has been added to the `Makefile` and should be
removed when the API does not require the build-tag anymore.

See-also: ceph/go-ceph#655
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-04-14 08:01:45 +00:00
Robert Vasek
8fb5739f21 build: more flexible handling of go build tags; added ceph_preview
ceph_preview tag is needed to make new go-ceph's RADOS read/write ops available

Signed-off-by: Robert Vasek <robert.vasek@cern.ch>
2022-03-27 19:24:26 +00:00
Niels de Vos
07e4084080 build: remove container-id file before building
In case building a new container-image fails, the old image has already
been removed by the same make target. The container-id file that is used
to prevent unneeded rebuilds, causes build problems in case the
container-image in the container-id file does not exist (anymore).

By removing the container-id file before rebuilding the image, there
should not be any issues on subsequent (attempted fixed) builds of the
container-images.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-02-04 10:23:58 +00:00
Madhu Rajanna
e743e06748 ci: install arch specific go in Dockerfile.test
Instead of installing the amd64 on all the
platforms, install architecture specific go
version in test dockerfile.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-12-15 05:13:36 +00:00
Madhu Rajanna
8a3fe53e87 ci: install arch specific go in Dockerfile.devel
Instead of installing the amd64 on all the
platforms, install architecture specific go
version for devel dockerfile

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-12-15 05:13:36 +00:00
Rakshith R
191b603974 ci: remove gh action gosec linter,since it is already part of golangci
This commit removes gosec standalone linter and related parts,
since golangci linter runs gosec linter too.

Signed-off-by: Rakshith R <rar@redhat.com>
2021-11-16 12:29:56 +01:00
Niels de Vos
97525f5e74 ci: add make go-test-api to GitHub Action
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-10-05 11:26:50 +00:00
Niels de Vos
5ea99fdd5b build: add yamlgen to build deployment files
This initial version of yamlgen generates deploy/scc.yaml based on the
deployment artifact that is provided by the new api/deploy/ocp package.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-10-05 11:26:50 +00:00
Niels de Vos
36e099d939 deploy: add API for getting OpenShift SecurityContextConstraints
This new Go module allows other projects (like Rook) to consume
deployment details (the SCC to get started) directly from Ceph-CSI.

Based-on: https://github.com/rook/rook/blob/3c93eb3/cluster/examples/kubernetes/ceph/operator-openshift.yaml#L47-L90
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-10-05 11:26:50 +00:00
Yati Padia
e077c1fdf5 cleanup: run codespell on containerized testing
This commit adds a new target codespell to the
make containerized-test.

Fixes: #2229

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-08-12 09:42:54 +05:30
Niels de Vos
538e36f7a7 ci: pass GITHUB_BASE_REF when running commitlint
GitHub Actions include a merge commit for the PR, which will defeat the
commitlint checking of all the commits inside the PR (only the merge
commit is checked).

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-06-07 18:19:23 +05:30
Niels de Vos
2f6fca0862 build: regenerate scripts/golangci.yml if build.env was updated
In case build.env was updated, scripts/golangci.yml needs to be
regenerated. It contains a reference to the build-tag that is used to
identify the Ceph version to link against. Failing to update the
scripts/golangci.yml configuration, may cause running tests fail.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-03-17 13:38:46 +00:00
Niels de Vos
92190dee5d ci: use "devel" branch instead of "master"
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-03-01 10:51:30 +05:30
Niels de Vos
61a16012b7 build: only use --cpuset options when the cgroup controller is available
Fixes: #1670
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-23 02:53:51 +00:00
Madhu Rajanna
cd96b3359b build: use CSI_IMAGE_VERSION in makefile
updated makefile to use CSI_IMAGE_VERSION
from build.env
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-09 17:14:19 +00:00
Madhu Rajanna
39b1f2b4d3 cleanup: fix mispell words
fixed mispell words in the repo.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-11-29 12:47:46 +05:30
Niels de Vos
cba466c163 ci: do not run "make mod-check" for containerized-tests
An updated CI job will run "make mod-check" in parallel with the full
containerized-test and containerized-build targets. This will hopefully
reduced the time that is needed for the whole
ci/centos/containerized-tests job.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-04 05:36:07 +00:00
Niels de Vos
afd6994e19 ci: allow passing USE_PULLED_IMAGE=yes to use pre-pulled images
When passing USE_PULLED_IMAGE=yes to the containerized-test or
containerized-build make targets, it is now possible to use pre-pulled
container images. This saves time as the container images will not get
created from scratch.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-04 05:36:07 +00:00
Prasanna Kumar Kalever
e9d5e731dc build: change default to use podman
With '# make image-cephcsi GOARCH=amd64'
On Fedora observed errors like:

 ---> Running in 3e2dbf48ebc6
OCI runtime create failed: this version of runc doesn't work on cgroups
v2: unknown
make: *** [Makefile:175: image-cephcsi] Error 1

hence it is recommended to switch to podman if available.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2020-08-19 11:52:10 +00:00
Niels de Vos
6d5456cc77 ci: use FETCH_HEAD like "commitlint --from" does
After fetching the target branch for the PR, the GIT_SINCE ref in the
git repository may not be set (in CI environments). This causes 'git
rebase' to fail. Use FETCH_HEAD instead, so that the checked out PR gets
rebased correctly.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-08-18 14:34:08 +00:00
Niels de Vos
104203ed03 ci: pass REBASE=1 to have commitlint rebase the branch
When Mergify adds a merge commit to the branch that is being tested with
commitlint, the tool tries to detect the most recent changes based on
the newly merged commit. This is for most PRs the master branch, and
that contains incorrect commit messages in the history. Because of this,
commitlint will fail.

By adding an option (REBASE=1) to the commitlint make target, CI jobs
can request a rebase so that the history of the PR becomes linear again
and commitlint should be able to detect only the new commits.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-08-18 14:34:08 +00:00
Niels de Vos
84b7e7b11c build: store detected container-cmd to prevent rebuilds
The 'need-container-cmd' make target is marked as .PHONY which will
cause it to be run every time. That triggers a rebuild of the container
images, even when that is not required.

By removing the 'need-container-cmd' target from .PHONY, and storing the
contents of CONTAINER_CMD in .container-cmd, unneeded rebuilds are
prevented.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-08-18 14:34:08 +00:00
Niels de Vos
9bd91bc9d3 ci: use last "git fetch" output to test commitlint
When running in the CI the git repository is not completely cloned. This
causes the 'commitlint' job to be unable to resolve the history of the
commits.

By using FETCH_HEAD, the last 'git fetch' output will be used.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-08-13 12:12:58 +05:30
Niels de Vos
35c5afcd3e ci: always fetch GIT_SINCE for commitlint
When running the commitlint CI job, the branch that the PR is based on
may not be available. That makes it impossible for commitlint to detect
the changes between the HEAD of the branch, and the commits in the PR.

By fetching GIT_SINCE unconditionally, commitlint should be able to
detect the changes and only run the tests against the commits that were
added through the PR.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-08-11 08:11:37 +00:00
Niels de Vos
74ba85f87b ci: add "make run-e2e"
The keeps the standard arguments for e2e testing in a single location
instead of spread over multiple files and CI jobs.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-31 16:03:54 +00:00
Madhu Rajanna
b949d5f9c7 build: Build and push multi architecture images to dockerhub
As quay.io doesnot support the multi architecture
images, We need to switch to dockerhub as it supports
multi-architecture images.

closes: #1003

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-16 08:57:38 +00:00
Niels de Vos
969b032f1c build: error out when Podman or Docker is not available
In case Podman or Docker is not installed, running containerized tests
or builds will fail with unclear errors.

Introduce a 'need-container-cmd' make target that checks if the
CONTAINER_CMD is available (and non-empty). When running without Podman
and Docker, the following error is displayed:

    $ make containerized-build
    cephcsi image settings: quay.io/cephcsi/cephcsi version canary
    Missing container support, install Podman or Docker
    make: *** [Makefile:135: need-container-cmd] Error 1

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-07 09:49:45 +00:00
Niels de Vos
f9f9fba26c ci: generate golangci.yml with correct CEPH_VERSION
When building against go-ceph, the most recent version of Ceph is
assumed to be available (currently Octopus). In case an older version of
the development packages is installed, building go-ceph will fail.

Golangci-lint does not accept the `-tags nautilus` parameter like other
Golang tools. Instead, the build-constraints need to be configured in a
confguration file.

This change takes care of the following:
- move the current scripts/golangci.yml to a template
- add the @@CEPH_VERSION@@ substitute
- generate the configuration file when needed

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-02 14:24:02 +00:00
Niels de Vos
c02413a663 build: Add -tags=nautilus to go test and go build
Build `GO_TAGS` based on the `CEPH_VERSION` from build.env. In case the
version is non-empty, pass `-tags=${CEPH_VERSION}` to any of the go
commandline and script that call go programs.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-06-30 07:55:37 +00:00
Yug Gupta
306d5b1da0 ci: add pylint as part of CI
add pylint to catch static issues of python
files in the repo.

User can now run make lint-py for pylint
check on python files.

Signed-off-by: Yug Gupta <ygupta@redhat.com>
2020-06-30 04:43:45 +00:00
Niels de Vos
44bf398c79 build: move GOARCH environment variable to targets that use it
This prevents some clutter while running 'make' commands on an
environment where 'go' is not installed. There is no requirement on 'go'
in the case containers are used for building and testing.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-06-28 17:46:37 +00:00
Niels de Vos
3cc791c7c4 ci: document TEST_COVERAGE and GO_COVER_DIR in buid.env
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-06-28 17:46:37 +00:00
Niels de Vos
4fd973b924 build: use BASE_IMAGE from build.env
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-06-28 17:46:37 +00:00
Niels de Vos
bc2ac1ccac build: move GOLANGCI_VERSION to build.env
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-06-28 17:46:37 +00:00
Niels de Vos
f83a065c8a build: move GOLANG_VERSION to build.env
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-06-28 17:46:37 +00:00
Niels de Vos
c0b53d3ad4 build: add option to compile e2e.test in a container
Run `make containerized-build TARGET=e2e.test` to build the e2e.test
executable in a container. This makes it possible for environments to
only have the dependencies for the runtime installed (most notably Ceph
shared libraries) and, run the e2e tests directly.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-06-02 08:28:54 +00:00
Niels de Vos
ed30fa19d4 build: use CURDIR instead of PWD
`make -C ${GOPATH/src/github.com/ceph/ceph-csi container...` commands
fail as the PWD variable gets substituted with the path where `make` is
called, not with the path of the Makefile that is passed with the -C
option.

The CURDIR variable is expended to the correct path, so `make -C ...`
works for all make targets now.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-05-31 15:03:58 +00:00
Niels de Vos
686db53fde build: prevent error message when 'go' is unavailable
In case the `go` executable is not available, any `make` command shows
the error `go: command not found` even when running the actual task in a
container (that will have `go` available).

So, redirect the error message to `/dev/null`. In case `go` is really
not available in the build environment, the `check-env` make target will
report it.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-05-19 07:31:58 +00:00
Niels de Vos
eff4f4ba50 ci: add commitlint target to the Makefile
When passing `GIT_SINCE` as an argument, the commits from then on
(defaults to origin/master) are used. A CI job can pass the base branch
or ID so a range of patches gets checked.

To test the last four commits, one can run:

    $ make containerized-test TARGET=commitlint GIT_SINCE=HEAD~4

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-05-15 18:20:29 +00:00
Madhu Rajanna
3482cb7091 ci: updated shell script to run individual test
Earlier we were running all the linter for non-go
files in one short, this wont be helpful for the
users who want to run particular tests.

now the Makefile as different target to
run separate lint test for different type
of non-go files.

Fixes: #979

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-05-14 16:54:19 +00:00
Niels de Vos
b89f7fa1d6 build: add check for functional environment
Updates: #1000
Reported-by: Madhu Rajanna <madhupr007@gmail.com>
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-05-12 10:52:59 +00:00
Yaniv Kaul
1f7d9760d5 Makefile: allow building with Podman
If Docker does not exist, use Podman.
Use all CPUs for the build, which hopefully will make it faster.

Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
Fixes: #987
2020-04-27 05:21:26 +00:00
Niels de Vos
d1e12cb0e2 add optional TARGET=.. parameter for 'make containerized-test'
Running the containerized-tests is relatively time consuming as it runs
all the tests. By adding a TARGET=.. option to the Makefile, it is now
possible to run a selected test, for example:

    make containerized-test TARGET=static-check

This reduces the time the tests take, so it is more friendly to the
developers while addressing problems.

When no TARGET=.. parameter is passed, the default tests are run.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-04-24 15:41:46 +00:00
Niels de Vos
32839948ef cleanup: move pkg/ to internal/
The internal/ directory in Go has a special meaning, and indicates that
those packages are not meant for external consumption. Ceph-CSI does
provide public APIs for other projects to consume. There is no plan to
keep the API of the internally used packages stable.

Closes: #903
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-04-23 11:00:59 +00:00
Niels de Vos
2c6c0decbe containerized-build: set WORKDIR and use in make target
By setting the WORKDIR in the container image, there is no need to pass
it on the commandline in the Makefile. This makes the line for the make
target a little cleaner.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-04-23 03:41:06 +00:00
Madhu Rajanna
bfa6064b4d Read baseimage from the dockerfile
Updated deploy.sh and build scripts
to read base image from the dockerfile

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-04-22 15:41:40 +00:00
Madhu Rajanna
4b540ffecc Update makefile and deploy.sh to pass build arguments
updated makefile to pass the required build
arguments for multistage docker build

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-04-22 15:41:40 +00:00
Madhu Rajanna
32e2a713e6 Fix multi architecture dockerfile
Add support for multi architecture build
for cephcsi. with multistage docker build
we we build cephcsi binary for both arm64
and amd64 architecture.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-04-22 15:41:40 +00:00