Commit Graph

1243 Commits

Author SHA1 Message Date
Madhu Rajanna
6786d92246 ci: remove kube 1.16 from Travis CI
As kube v1.18 is released we dont need to test
-2 version in our CI, removing the 1.16 testing
for the same reason.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-10 09:47:21 +00:00
Madhu Rajanna
f05c9a6a93 ci: fix psp issue in minikube latest version
With minikube versions greater than 1.6.2 and less than 1.11.1, the YAML files
minikube path will not be automatically applied to the cluster. we will get
errors during bootstrap of the cluster if the admission controller is enabled.

To use Pod Security Policies with these versions of minikube, first start a
cluster without the `PodSecurityPolicy` admission controller enabled.

Next, apply the psp yaml. and stop the cluster and then restart it
with the admission controller enabled.

```
minikube start
kubectl apply -f /path/to/psp.yaml
minikube stop
minikube start --extra-config=apiserver.enable-admission-plugins=PodSecurityPolicy
```

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-10 09:47:21 +00:00
Madhu Rajanna
8aae7b553c ci: update minikube version to latest release
updated minikube to latest release v1.12.0

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-10 09:47:21 +00:00
Yug
1490daed7e cleanup: Avoid usage of numbers
Add seperate functions to handle all
levels and types of logging.

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-07-10 07:41:23 +00:00
Yug
8dc4ab6b1b rebase: update k8s.io/klog to v2.3.0
Update klog version to v2.3.0

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-07-10 07:41:23 +00:00
Madhu Rajanna
b773e821c8 ci: use testCephFS flag to skip cephfs tests
This commit fixes the flag check to skip
cephfs E2E tests.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-09 13:50:21 +00:00
Sven Anderson
13f291dfc6 cleanup: use errors.As() for error type checks
Replaces some remaining old-style error type checks with errors.As()

Signed-off-by: Sven Anderson <sven@redhat.com>
2020-07-09 09:58:33 +00:00
Madhu Rajanna
310c36e319 ci: separate the cephfs and rbd tests
As the maximum time allocated for the Travis
CI is 50 minutes and the cephfs and rbd E2E
takes around 45 Minutes to run completely.
This is blocking us from adding more tests in E2E.
splitting out the E2E will help us to run more tests
for each driver as we will get 50 minutes for
each Travis CI instance.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-09 06:42:28 +00:00
Madhu Rajanna
9d1c91f3c0 ci: Pass on the arguments to E2E
Pass on the arugments which are sent when
calling the scripts to the E2E.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-09 06:42:28 +00:00
Madhu Rajanna
63b795d165 e2e: provide an option to skip rbd and cephfs e2e
Added two new parameters for e2e test to skip
rbd and cephfs tests. This will help us to
run more test in Travis CI.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-09 06:42:28 +00:00
Madhu Rajanna
4e82384b43 rbd: check image not found error
during the checkSnapCloneExists we are checking
the image, if the image not found we are deleting
the snapshot on the parent image, This PR corrects
the comparasion. instead of snapshotNotFound we need
to check ImageNotFound error.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-08 17:09:59 +00:00
Mudit Agarwal
953cfa508a util: fix tracevol.py to work with volumeNamePrefix in storageclass
tarcevol.py works with a fixed volume name prefix "csi-vol-", this could be changed by the storageclass
and hence tracevol.py should work with the dynamic value rather than the static.

Signed-off-by: Mudit Agarwal <muagarwa@redhat.com>
2020-07-08 14:33:15 +00:00
Mudit Agarwal
2ef5729a28 cleanup: restructure a part of existing code
Have restructered a part of existing code so that it would be useful to get pvdata

Signed-off-by: Mudit Agarwal <muagarwa@redhat.com>
2020-07-08 14:33:15 +00:00
Madhu Rajanna
82080dcfa9 util: update unit testing for deep flatten
updated unit testing for the kernel check
for deep flatten feature for both supported
upstream kernel version (5.1.0+) and RHEL
8.2 backport

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-08 05:48:42 +00:00
Niels de Vos
969b032f1c build: error out when Podman or Docker is not available
In case Podman or Docker is not installed, running containerized tests
or builds will fail with unclear errors.

Introduce a 'need-container-cmd' make target that checks if the
CONTAINER_CMD is available (and non-empty). When running without Podman
and Docker, the following error is displayed:

    $ make containerized-build
    cephcsi image settings: quay.io/cephcsi/cephcsi version canary
    Missing container support, install Podman or Docker
    make: *** [Makefile:135: need-container-cmd] Error 1

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-07 09:49:45 +00:00
Yug
34593c9af0 cleanup: Reduce cyclomatic complexity of topology_test
Reduce redundant code to make the test simpler.

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-07-07 06:08:55 +00:00
Madhu Rajanna
f05c8ed2a7 journal: remove unused rados CLI functions
Removed unused rados CLI functions, as rados
CLI is replaced with go-ceph bindings.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-06 15:03:44 +00:00
Madhu Rajanna
9bb79fbe7d journal: replace getStoredImageId with go-ceph
moved implementation of getStoredImageID to go-ceph
to get better performance.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-06 15:03:44 +00:00
Madhu Rajanna
341a3c9f59 journal: replace rados CLI with go-ceph to storeimage id
Moved the implementation of storeImageID from rados
CLI to go-ceph api.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-06 15:03:44 +00:00
Madhu Rajanna
8f758450d8 rbd: add RHEL 8.2 kernel to the list
as RHEL 8.2 supports the deep-flatten
feature, added it to the list to map
the rbd images on the node without flattening.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-06 12:20:00 +00:00
Madhu Rajanna
04c8c7fd4a rbd: correct upstream kernel version for deep-flatten
as v5.1.0 supports the deep-flatten feature,lowering
the required version to map rbd images which
are having deep-flatten feature

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-06 12:20:00 +00:00
Madhu Rajanna
6838d30aff e2e: reduce the pod cound and wait time
Reduced the number of pods created
in ROX E2E to save some time in E2E
and changed the waiting time from 2 to 1
min.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-06 10:08:31 +00:00
Madhu Rajanna
b76d470bf1 e2e: increase e2e test timeout
as we are creating more resources
the e2e timeout is not enough we need
to increase the e2e timeout to higher
value for now.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-06 10:08:31 +00:00
Madhu Rajanna
db1b5f9c55 e2e: move deploy timeout to build.env
moved deploy timeout from travis
scripts to build.env

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-06 10:08:31 +00:00
Madhu Rajanna
5838b08edd e2e: Move e2e timeout to build.env
Moved e2e test timeout to build.env

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-06 10:08:31 +00:00
Madhu Rajanna
032fb228dd rbd: take lock on parent image during snapshotcreate
we need to take lock on parent rbd image when
we are creating a snapshot from it, if the user
tries to delete/resize the rbd image when we are
taking snapshots,we may face issues. if the volume
lock is present on the rbd image, the user cannot
resize the rbd image nor delete the rbd image.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-06 10:08:31 +00:00
Madhu Rajanna
f946efb7b7 e2e: set maxsnapshotsonimage during helm install
as we need to test the maxsnapshotsonimage we
need to set the limit to minimal value which we
can test in CI as the default limit is 450,which
cannot be tested in CI.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-06 10:08:31 +00:00
Madhu Rajanna
61a81d35e8 rbd: flatten cloned images to freeup snapshot
flatten cloned images to remove the link
with the snapshot as the parent snapshot can
be removed from the trash.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-06 10:08:31 +00:00
Madhu Rajanna
8ef7143e6c rbd: add maxsnapshotsonimage flag
Added maxsnapshotsonimage flag to flatten
the older rbd images on the chain to avoid
issue in krbd.The limit is in krbd since it
only allocate 1 4KiB page to handle all the
snapshot ids for an image.

The max limit is 510 as per
https://github.com/torvalds/linux/blob/
aaa2faab4ed8e5fe0111e04d6e168c028fe2987f/drivers/block/rbd.c#L98
in cephcsi we arekeeping the default to 450 to reserve 10%
to avoid issues.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-06 10:08:31 +00:00
Madhu Rajanna
c04b903eca rbd: add listsnapshots function
added listsnapshots function for an
rbd image to list all the snapshots
created from an rbd images, This will
list the snapshots which are in trash also.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-06 10:08:31 +00:00
Sven Anderson
de74c36d8c cleanup: address err113 warnings about direct error comparisons
Several places in the code compared errors directly with the go-ceph
sentinel errors.  This change uses the errors.Is() function of go
1.13 instead.  The err113 linter reported this issue as:

err113: do not compare errors directly, use errors.Is() instead

Signed-off-by: Sven Anderson <sven@redhat.com>
2020-07-06 08:57:46 +00:00
Niels de Vos
7affb9289d ci: display a warning when GO_ARCH is not set for image-cephcsi
`make image-cephcsi` will fail when Golang is not installed. There is no
strict requirement for Golang to be available, it is only used to gather
the architecture of the OS where the image is built. It is possible to
build the image successfully with `make image-cephcsi GOARCH=amd64`.

In case Golang is not installed, GOARCH can not be detected
automatically. This will cause a failure while installing Golang in the
container image. Because the failure is not very clear, display a
warning in the case the GO_ARCH (from ${GOARCH} in the Makefile) is not
set.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-06 06:20:53 +00:00
Madhu Rajanna
826f7126cd doc: update doc to delete alpha snapshot
updated upgrade documentation to remove
the snapshot created by alpha driver before
upgrade of CSI driver as beta snapshot is not
backward compatible with the alpha snapshot.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-03 11:32:00 +00:00
Niels de Vos
267c709194 cleanup: use errors.As() in rbd/controllerserver methods
See-also: https://github.com/golang/go/wiki/ErrorValueFAQ#how-should-i-change-my-error-handling-code-to-work-with-the-new-features
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-03 09:12:48 +00:00
Niels de Vos
91ade42a32 cleanup: use errors.As() in journal.Connection methods
See-also: https://github.com/golang/go/wiki/ErrorValueFAQ#how-should-i-change-my-error-handling-code-to-work-with-the-new-features
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-03 09:12:48 +00:00
Niels de Vos
ad04e0d8c4 cleanup: use errors.As() in cephfs.checkVolExists()
See-also: https://github.com/golang/go/wiki/ErrorValueFAQ#how-should-i-change-my-error-handling-code-to-work-with-the-new-features
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-03 09:12:48 +00:00
Niels de Vos
72c74f5f8a cleanup: use errors.As() in cephfs.NodeStageVolume()
See-also: https://github.com/golang/go/wiki/ErrorValueFAQ#how-should-i-change-my-error-handling-code-to-work-with-the-new-features
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-03 09:12:48 +00:00
Niels de Vos
c9d5bc8a08 cleanup: use errors.As() in e2e/utils
See-also: https://github.com/golang/go/wiki/ErrorValueFAQ#how-should-i-change-my-error-handling-code-to-work-with-the-new-features
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-03 09:12:48 +00:00
Niels de Vos
a47cfbc6c8 cleanup: use errors.As() in rbdVolume.Exists() and rbd.checkSnapExists()
See-also: https://github.com/golang/go/wiki/ErrorValueFAQ#how-should-i-change-my-error-handling-code-to-work-with-the-new-features
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-03 09:12:48 +00:00
Niels de Vos
d4dad7c189 cleanup: use errors.As() in rbd.NodeUnstageVolume()
See-also: https://github.com/golang/go/wiki/ErrorValueFAQ#how-should-i-change-my-error-handling-code-to-work-with-the-new-features
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-03 09:12:48 +00:00
Niels de Vos
7126e92b4f cleanup: use errors.As() in rbd.rbdStatus()
See-also: https://github.com/golang/go/wiki/ErrorValueFAQ#how-should-i-change-my-error-handling-code-to-work-with-the-new-features
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-03 09:12:48 +00:00
Niels de Vos
f93730dee7 cleanup: use errors.As() to compare errors in cephfs/controllerserver
See-also: https://github.com/golang/go/wiki/ErrorValueFAQ#how-should-i-change-my-error-handling-code-to-work-with-the-new-features
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-03 09:12:48 +00:00
Niels de Vos
57d0989593 cleanup: use errors.As() to compare errors in util/cephcmds
See-also: https://github.com/golang/go/wiki/ErrorValueFAQ#how-should-i-change-my-error-handling-code-to-work-with-the-new-features
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-03 09:12:48 +00:00
Niels de Vos
8c4379862a rebase: remove "github.com/pkg/errors" from go.mod
There is no direct dependency on "github.com/pkg/errors" anymore, so it
can be removed from go.mod.

After running `go mod tidy`, the "github.com/pkg/errors" gets downgraded
to a version that is referenced in other depndencies. This was
unexpected, but seems needed.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-03 09:12:48 +00:00
Niels de Vos
e393e298d0 doc: do not mention "github.com/pkg/errors"
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-03 09:12:48 +00:00
Niels de Vos
92aae4834e cleanup: use standard Golang errors package
"github.com/pkg/errors" does not offer more functionlity than that we
need from the standard "errors" package. With Golang v1.13 errors can be
wrapped with `fmt.Errorf("... %w", err)`. `errors.Is()` and
`errors.As()` are available as well.

See-also: https://tip.golang.org/doc/go1.13#error_wrapping
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-03 09:12:48 +00:00
Niels de Vos
8effa0cd3e util: add Unwrap() to error types
See-also: https://github.com/golang/go/wiki/ErrorValueFAQ#i-have-a-type-that-implements-error-and-holds-a-nested-error-how-should-i-adapt-it-to-the-new-features
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-03 09:12:48 +00:00
Niels de Vos
6d6b5bab3c rbd: add Unwrap() to error types
See-also: https://github.com/golang/go/wiki/ErrorValueFAQ#i-have-a-type-that-implements-error-and-holds-a-nested-error-how-should-i-adapt-it-to-the-new-features
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-03 09:12:48 +00:00
Niels de Vos
fad0cf7f1c cephfs: add Unwrap() to error types
See-also: https://github.com/golang/go/wiki/ErrorValueFAQ#i-have-a-type-that-implements-error-and-holds-a-nested-error-how-should-i-adapt-it-to-the-new-features
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-03 09:12:48 +00:00
Niels de Vos
0bafa2be8c doc: add comments to rbd error methods
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-03 09:12:48 +00:00