In case build.env was updated, scripts/golangci.yml needs to be
regenerated. It contains a reference to the build-tag that is used to
identify the Ceph version to link against. Failing to update the
scripts/golangci.yml configuration, may cause running tests fail.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
An updated CI job will run "make mod-check" in parallel with the full
containerized-test and containerized-build targets. This will hopefully
reduced the time that is needed for the whole
ci/centos/containerized-tests job.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
When passing USE_PULLED_IMAGE=yes to the containerized-test or
containerized-build make targets, it is now possible to use pre-pulled
container images. This saves time as the container images will not get
created from scratch.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
With '# make image-cephcsi GOARCH=amd64'
On Fedora observed errors like:
---> Running in 3e2dbf48ebc6
OCI runtime create failed: this version of runc doesn't work on cgroups
v2: unknown
make: *** [Makefile:175: image-cephcsi] Error 1
hence it is recommended to switch to podman if available.
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
After fetching the target branch for the PR, the GIT_SINCE ref in the
git repository may not be set (in CI environments). This causes 'git
rebase' to fail. Use FETCH_HEAD instead, so that the checked out PR gets
rebased correctly.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
When Mergify adds a merge commit to the branch that is being tested with
commitlint, the tool tries to detect the most recent changes based on
the newly merged commit. This is for most PRs the master branch, and
that contains incorrect commit messages in the history. Because of this,
commitlint will fail.
By adding an option (REBASE=1) to the commitlint make target, CI jobs
can request a rebase so that the history of the PR becomes linear again
and commitlint should be able to detect only the new commits.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
The 'need-container-cmd' make target is marked as .PHONY which will
cause it to be run every time. That triggers a rebuild of the container
images, even when that is not required.
By removing the 'need-container-cmd' target from .PHONY, and storing the
contents of CONTAINER_CMD in .container-cmd, unneeded rebuilds are
prevented.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
When running in the CI the git repository is not completely cloned. This
causes the 'commitlint' job to be unable to resolve the history of the
commits.
By using FETCH_HEAD, the last 'git fetch' output will be used.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
When running the commitlint CI job, the branch that the PR is based on
may not be available. That makes it impossible for commitlint to detect
the changes between the HEAD of the branch, and the commits in the PR.
By fetching GIT_SINCE unconditionally, commitlint should be able to
detect the changes and only run the tests against the commits that were
added through the PR.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
The keeps the standard arguments for e2e testing in a single location
instead of spread over multiple files and CI jobs.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
As quay.io doesnot support the multi architecture
images, We need to switch to dockerhub as it supports
multi-architecture images.
closes: #1003
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
In case Podman or Docker is not installed, running containerized tests
or builds will fail with unclear errors.
Introduce a 'need-container-cmd' make target that checks if the
CONTAINER_CMD is available (and non-empty). When running without Podman
and Docker, the following error is displayed:
$ make containerized-build
cephcsi image settings: quay.io/cephcsi/cephcsi version canary
Missing container support, install Podman or Docker
make: *** [Makefile:135: need-container-cmd] Error 1
Signed-off-by: Niels de Vos <ndevos@redhat.com>
When building against go-ceph, the most recent version of Ceph is
assumed to be available (currently Octopus). In case an older version of
the development packages is installed, building go-ceph will fail.
Golangci-lint does not accept the `-tags nautilus` parameter like other
Golang tools. Instead, the build-constraints need to be configured in a
confguration file.
This change takes care of the following:
- move the current scripts/golangci.yml to a template
- add the @@CEPH_VERSION@@ substitute
- generate the configuration file when needed
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Build `GO_TAGS` based on the `CEPH_VERSION` from build.env. In case the
version is non-empty, pass `-tags=${CEPH_VERSION}` to any of the go
commandline and script that call go programs.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
add pylint to catch static issues of python
files in the repo.
User can now run make lint-py for pylint
check on python files.
Signed-off-by: Yug Gupta <ygupta@redhat.com>
This prevents some clutter while running 'make' commands on an
environment where 'go' is not installed. There is no requirement on 'go'
in the case containers are used for building and testing.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Run `make containerized-build TARGET=e2e.test` to build the e2e.test
executable in a container. This makes it possible for environments to
only have the dependencies for the runtime installed (most notably Ceph
shared libraries) and, run the e2e tests directly.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
`make -C ${GOPATH/src/github.com/ceph/ceph-csi container...` commands
fail as the PWD variable gets substituted with the path where `make` is
called, not with the path of the Makefile that is passed with the -C
option.
The CURDIR variable is expended to the correct path, so `make -C ...`
works for all make targets now.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
In case the `go` executable is not available, any `make` command shows
the error `go: command not found` even when running the actual task in a
container (that will have `go` available).
So, redirect the error message to `/dev/null`. In case `go` is really
not available in the build environment, the `check-env` make target will
report it.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
When passing `GIT_SINCE` as an argument, the commits from then on
(defaults to origin/master) are used. A CI job can pass the base branch
or ID so a range of patches gets checked.
To test the last four commits, one can run:
$ make containerized-test TARGET=commitlint GIT_SINCE=HEAD~4
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Earlier we were running all the linter for non-go
files in one short, this wont be helpful for the
users who want to run particular tests.
now the Makefile as different target to
run separate lint test for different type
of non-go files.
Fixes: #979
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
If Docker does not exist, use Podman.
Use all CPUs for the build, which hopefully will make it faster.
Signed-off-by: Yaniv Kaul <ykaul@redhat.com>
Fixes: #987
Running the containerized-tests is relatively time consuming as it runs
all the tests. By adding a TARGET=.. option to the Makefile, it is now
possible to run a selected test, for example:
make containerized-test TARGET=static-check
This reduces the time the tests take, so it is more friendly to the
developers while addressing problems.
When no TARGET=.. parameter is passed, the default tests are run.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
The internal/ directory in Go has a special meaning, and indicates that
those packages are not meant for external consumption. Ceph-CSI does
provide public APIs for other projects to consume. There is no plan to
keep the API of the internally used packages stable.
Closes: #903
Signed-off-by: Niels de Vos <ndevos@redhat.com>
By setting the WORKDIR in the container image, there is no need to pass
it on the commandline in the Makefile. This makes the line for the make
target a little cleaner.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Add support for multi architecture build
for cephcsi. with multistage docker build
we we build cephcsi binary for both arm64
and amd64 architecture.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
`go mod verify` does not fail when all moduled under vendor/ are
updated. However it does update `go.mod` in case it does not reflect the
downloaded versions.
By adding this additional verification in `make mod-check`, future
updates to vendor/ will require a correct go.mod as well.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
`make containerized-test` has been added as a make target. This runs the
'make test' target in a container. All test dependencies are installed
in the container once, and the container is kept around for running
`make containerized-test` subsequently.
The test container is based on Fedora:latest so that all test tools get
easily installed and are available in a recent version. The production
container is based on the Ceph container which has CentOS as Operating
System and therefor a more stable (too old) toolset.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
This PR adds the support for helm
installation, and cephcsi helm charts
deployment and teardown and also runs E2E
on for helm charts.
Add socat to provide port forwadring access for helm
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
This makes it possible to build on any platform that supports Linux
containers. The container image used for building is created once, or on
updating the `scripts/Dockerfile.build` and is cached afterwards.
To build the executable in a container, use `make containerized-build`
and everything will be done automatically. The executable will also be
available on the usual location.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Update CI merge job to build and push Arm64 image to
quay.io/cephcsi/cephcsi:version-arm64.
Add CI PR job running on Travis Arm64 nodes to make sure cephcsi
compiles successfully on Arm64.
No CI test job is availabe for Arm64 now due to below issues
- k8s-csi sidecar images for Arm64 are not available
- Travis Arm64 CI job runs inside unprivileged LXD which blocks
launching minikube test environment
Signed-off-by: Yibo Cai <yibo.cai@arm.com>
update driver version and add git commit
to the image. This will help us to identify
what latest git commit image contains.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
update travis and makefile for functional test
skip docker pull if image is already present
on local machine.
if the image is not present locally pull the
image from repo.
export kubeconfig in travis
build cephcsi image in travis job for
functional testing
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
currently, we have 3 docker files(cephcsi,rbd,cephfs) in the ceph-csi repo.
[commit ](85e121ebfe)
added by John to build a single image which can act as rbd or
cephfs based on the input configuration.
This PR updates the makefile and kubernetes templates to use
the unified image and also its deletes the other two dockerfiles.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
