Commit Graph

1653 Commits

Author SHA1 Message Date
Niels de Vos
3cc791c7c4 ci: document TEST_COVERAGE and GO_COVER_DIR in buid.env
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-06-28 17:46:37 +00:00
Niels de Vos
4fd973b924 build: use BASE_IMAGE from build.env
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-06-28 17:46:37 +00:00
Niels de Vos
f1da7d9bd1 ci: move minikube variables to build.env
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-06-28 17:46:37 +00:00
Niels de Vos
21946ae345 ci: move HELM_VERSION to build.env
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-06-28 17:46:37 +00:00
Niels de Vos
079459c247 build: move GOSEC_VERSION to build.env
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-06-28 17:46:37 +00:00
Niels de Vos
bc2ac1ccac build: move GOLANGCI_VERSION to build.env
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-06-28 17:46:37 +00:00
Niels de Vos
33dc861f23 ci: use "gimme" to install Golang in Travis CI
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-06-28 17:46:37 +00:00
Niels de Vos
f83a065c8a build: move GOLANG_VERSION to build.env
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-06-28 17:46:37 +00:00
Yug Gupta
c9ad904331 cephfs: Add support for multiple subvolumegroups
With the current code base, the subvolumegroup will
be created once, and even for a different cluster,
subvolumegroup creation is not allowed again.

Added support multiple subvolumegroups creation by
validating one subvolumegroup creation per cluster.

Fixes: #1123

Signed-off-by: Yug Gupta <ygupta@redhat.com>
2020-06-26 09:19:24 +00:00
Niels de Vos
efd9b31043 cleanup: use util.MountOptionsAdd() in CephFS volumemounter
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-06-24 13:33:44 +00:00
Niels de Vos
cdbc56de7d ci: add test for MountOptionsAdd()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-06-24 13:33:44 +00:00
Niels de Vos
887aab1d72 util: add MountOptionsAdd() to add mount options
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-06-24 13:33:44 +00:00
John Mulligan
75088aa36d util: add doc comments for exported functions in errors.go
A number of exported functions in errors.go were missing doc comments.
Add them.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-06-22 15:41:00 +00:00
John Mulligan
09e1c856d0 journal: do not return errors from remove omap func if omap missing
The previous function used to remove omap keys apparently did not
return errors when removing omap keys from a missing omap (oid).
Mimic that behavior when using the api.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-06-22 15:41:00 +00:00
John Mulligan
d32cef5b9f ci: remove comment referring to a function that no longer exists
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-06-22 15:41:00 +00:00
John Mulligan
c14cbee132 journal: change omap set func to handle multiple key-value pairs
For any function that sets more than one key on a single oid setting
them as a batch will be more efficient.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-06-22 15:41:00 +00:00
John Mulligan
a8409eccc8 journal: change omap remove func to handle multiple keys at once
For any function that removes more than one key on a single oid removing
them as a batch will be more efficient.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-06-22 15:41:00 +00:00
John Mulligan
0ac5f40d09 journal: change omap get func to handle multiple keys at once
Taking this appraoch means that any function that must get more than one
key's value from the same oid can be more efficient by calling out to
ceph only once.

To be cautious and avoid missing things we always request ceph return
more keys than we actually expect to be set on the oid. If there are
unexpected keys there, we will not miss the keys we want if we first hit
an unexpected key if we were to limit ourselves to iterating only over
the number of keys we're expecting to be on the object.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-06-22 15:41:00 +00:00
John Mulligan
cd24bb3f5c journal: convert journal to use new omap functions
Convert the business-logic of the journal to use the new go-ceph based
omap manipulation functions.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-06-22 15:41:00 +00:00
John Mulligan
0b99bdaa82 journal: add omap.go with go-ceph based omap manipulation funcs
These new omap manipulation functions (get/set/remove) are roughly
equivalent to the previous command-line based approach but rely
on direct api calls to ceph.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-06-22 15:41:00 +00:00
John Mulligan
f3192bd1b3 util: add New* funcs for ErrKeyNotFound & ErrPoolNotFound types
These types have private fields but we need to construct them outside of
the util package. Add New* methods for both.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-06-22 15:41:00 +00:00
Madhu Rajanna
1a1ad11f57 doc: update coding doc to correct import order
Updated coding doc to correct the import order
as per the standard. More info can be found on
https://github.com/golang/go/wiki/CodeReviewComments#imports

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-06-22 09:51:23 +00:00
Madhu Rajanna
03947ebcc5 doc: update readme to add ROX support
Updated Readme to add ROX support
for rbd PVC.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-06-22 06:15:40 +00:00
Madhu Rajanna
39561b9675 e2e: Add E2E to mount rbd PVC as ro in app
Added an E2E to mount rbd PVC as readonly
in application pod and try to create some
file in Readonly PVC,when we try to create
files on RO PVC, we should get error.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-06-22 06:15:40 +00:00
Madhu Rajanna
649aeb7aaf rbd: Add support for rbd ROX PVC mounting
if the PVC access mode is ReadOnlyMany
or single node readonly, mounting the rbd
device path to the staging path as readonly
to avoid the write operation.

If the PVC acccess mode is readonly, mapping
rbd images as readonly.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-06-22 06:15:40 +00:00
Niels de Vos
da40d8e05e rbd: use librbd.FeatureSet for features
go-ceph v0.3 adds constants for ImageFeature values and their names.
Instead of hardcoding "layering" in several places, use the constant
given by librbd.

The rbdVolume.ImageFeatures does not seem to be used anywhere after the
conversion. Stashing the image metadata does include the ImageFeatures
as these are retrieved when getting the image information. It is safe to
drop ImageFeatures altogether and only use the imageFeatureSet instead.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-06-19 11:37:29 +00:00
Niels de Vos
decfc1ae2c rbd: make hasSnapshotFeature() a method of rbdVolume
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-06-19 11:37:29 +00:00
Humble Chirammal
dcbf71dd5d cleanup: avoid double import of kubernetes library in e2e utils
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-06-19 08:42:08 +00:00
Mudit Agarwal
e9082768b6 cephfs: enhance tracevol.py script to work with cephfs
tracevol.py script traces a RBD PVC to its RADOS map, key and image.
The script is enhanced to provide the same functionality for cephfs now.

Signed-off-by: Mudit Agarwal <muagarwa@redhat.com>
2020-06-19 05:48:27 +00:00
Mudit Agarwal
78963737d3 rbd: fix tracevol.py script to work with python 3.x
tracevol.py script traces a RBD PVC to its RADOS map, key and image.
It needs minor fixup in order to work with python 3.x

Signed-off-by: Mudit Agarwal <muagarwa@redhat.com>
2020-06-19 05:48:27 +00:00
Niels de Vos
ebe378c79a util: trim "\x00" characters from the kernel version
It seems that convering the release component from the unix.Utsrelease
type leaves some trailing "\x00" characters.

While splitting the string to compare kernel versions, these additional
characters might prevent converting the string to an int. Strip the
additional characters before returning the string.

Note:
  "\x00" characters are not visible when printing to a file or screen.
  They can be seen in hex-editors, or sending the output through 'xxd'.

Fixes: #1167
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-06-18 11:42:47 +00:00
Niels de Vos
1b0b26a7a6 util: log kernel version on start
It is useful to have the kernel version logged while starting binaries.
Some functionality depends on the version of the kernel, debugging
issues related to this will be easier.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-06-18 11:42:47 +00:00
Niels de Vos
209a5e5602 util: add function to get the kernel version
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-06-18 11:42:47 +00:00
Madhu Rajanna
2dc0bffc0a e2e: Remove arm64 build test from CI
as we are building the cephcsi inside
a container for both amd64 and arm64 we
dont need to have a separate E2E to
test the build on arm64 machine.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-06-18 09:21:48 +00:00
Niels de Vos
3364fe7b78 rbd: remove unneeded updateVolWithImageInfo()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-06-17 09:28:57 +00:00
Niels de Vos
824f38daaa rbd: use go-ceph to implement getImageInfo()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-06-17 09:28:57 +00:00
Niels de Vos
772d1dfa77 rebase: use go-ceph v0.3.0
v0.3.0 adds support for rbd.FeatureSet that can be used to parse the
features of an RBD image. This will be used in the followup commit that
adds rbdVolume.getImageInfo().

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-06-17 09:28:57 +00:00
Humble Chirammal
323cc0e3bb cleanup: avoid comparing errors directly
Go 1.13 contains support for error wrapping. To support wrapping,
fmt.Errorf now has a %w verb for creating wrapped errors, and three
new functions in the errors package ( errors.Unwrap, errors.Is and
errors.As) simplify unwrapping and inspecting wrapped errors.

With this change, If we currently compare errors using ==, we have to
use errors.Is instead. Example:

if err == io.ErrUnexpectedEOF

becomes

if errors.Is(err, io.ErrUnexpectedEOF)

https://tip.golang.org/doc/go1.13#error_wrapping

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-06-17 06:26:11 +00:00
John Mulligan
e111f2b613 scripts: add -mod=vendor to go run in check-env.sh
Without -mod=vendor running go run in this script may take more
resources than needed to execute. This also makes it consistent go build
(and alike) are invoked in the other scripts and the Makefile.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-06-17 04:58:32 +00:00
Humble Chirammal
134e11e26e doc: correct PVC expansion documentation
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-06-16 10:31:29 +00:00
Yug Gupta
5fbc153e0f cephfs: fix IneffAssign warnings in resizeVolume()
IneffAssign warns about the two following statements:

Line 147: warning: ineffectual assignment to supported (ineffassign)
Line 148: warning: ineffectual assignment to ok (ineffassign)

Reported-by: https://goreportcard.com/report/github.com/ceph/ceph-csi
Updates: #975

Signed-off-by: Yug Gupta <ygupta@redhat.com>
2020-06-16 09:06:18 +00:00
Humble Chirammal
2793b79ad8 doc: Correct PVC mention in the deployment doc
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-06-11 10:53:10 +00:00
Humble Chirammal
d11266e255 cleanup: remove double import of csi spec from nodeserver
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-06-11 10:28:24 +00:00
Humble Chirammal
333548b05c doc: Add v2.1.2 release details to release table
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-06-11 05:46:58 +00:00
Humble Chirammal
c1a269191c doc: make sure configmap object referred or documented correctly
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-06-10 11:33:10 +00:00
Yug Gupta
8b606f538f build: add check for ceph development headers
The script checks for the ceph development headers.
In case required packages are not found, script
suggests to run containerized build.

Signed-off-by: Yug Gupta <ygupta@redhat.com>
2020-06-10 09:49:55 +00:00
Niels de Vos
be703d1e42 util: implement CreateObject() with go-ceph
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-06-10 07:20:25 +00:00
Niels de Vos
6e24b10364 util: implement RemoveObject() with go-ceph
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-06-10 07:20:25 +00:00
Niels de Vos
926f1e813c rbd: keep rados.IOContext for re-use
This prevents the need to open the IOContext for additional operations
on the image.

It also addresses a leak of the IOContext in case `rbdVolume.open()` was
called. The method only returned the `rbd.Image` without the possibility
to close the related IOContext.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-06-09 13:17:31 +00:00
Niels de Vos
81b17c5157 ci: pass target filename of /etc/resolv.conf in case it is a symlink
In some Linux distributions the /etc/resolv.conf file is a symlink. This
file gets included in the Kubernetes containers and will be used for
resolving hostnames. By including the symlink, it is possible that that
target file is not available in the container(s). This will cause
problems when resolving hostnames, and Kubernetes will not get deployed.

The default minikube VM provides /run/systemd/resolve/resolv.conf, with
/etc/resolv.conf being a symlink. Therefor, it is needed to pass the
`--extra-config=kubelet.resolv-conf=..` parameter to `kubeadm`.

In case minikube is started with `--vm-driver=none` and
/run/systemd/resolve/resolv.conf does not exist, the local
/etc/resolv.conf will be used for inclusion in the Kubelet container. If
this is a symlink, the final destination should get passed with
`--extra-config=kubelet.resolv-conf=..` so that a working hostname
resolution configuration is available in the container.

Updates: #1121
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-06-05 08:57:19 +00:00