Commit Graph

2910 Commits

Author SHA1 Message Date
Yug
91662b54b6 e2e: move yaml filepath declaration to global
declared yaml filepath in gobal for reusing.

Co-authored-by: Madhu Rajanna <madhupr007@gmail.com>
Signed-off-by: Yug <yuggupta27@gmail.com>
2021-06-10 21:46:53 +00:00
Yug
30092ff4e1 e2e: add an option to set rbd storageclass name
added an option to set storageclass name.

Co-authored-by: Madhu Rajanna <madhupr007@gmail.com>
Signed-off-by: Yug <yuggupta27@gmail.com>
2021-06-10 21:46:53 +00:00
Yug
76e9b4b02a e2e: add function to create rbd pool
added a new function to create rbd pool
in e2e.

Co-authored-by: Madhu Rajanna <madhupr007@gmail.com>
Signed-off-by: Yug <yuggupta27@gmail.com>
2021-06-10 21:46:53 +00:00
Yug
e302797217 e2e: update listRBDImages to list images in different pool
made pool as a argument of listRBDImages to support
listing of rbd images in different pools.

Co-authored-by: Madhu Rajanna <madhupr007@gmail.com>
Signed-off-by: Yug <yuggupta27@gmail.com>
2021-06-10 21:46:53 +00:00
Yug
d992803e9e rbd: Update pool name in image chain
While traversing image chain, the parent
image can be present in a different pool
that the one child is in. So, updating
pool name in the next itteration to
that of the Parent.

Co-authored-by: Madhu Rajanna <madhupr007@gmail.com>
Signed-off-by: Yug <yuggupta27@gmail.com>
2021-06-10 21:46:53 +00:00
Yug
1f6a9cabfd rbd: verify if pool name is not empty
Validate Snapshot request to check if the
passed pool name is not empty.

Co-authored-by: Madhu Rajanna <madhupr007@gmail.com>
Signed-off-by: Yug <yuggupta27@gmail.com>
2021-06-10 21:46:53 +00:00
Yug
3898ae34a7 rbd: open new ioctx connection
if the parent and child clones are in
different namespaces we need to open a new
ioctx for pools.

Co-authored-by: Madhu Rajanna <madhupr007@gmail.com>
Signed-off-by: Yug <yuggupta27@gmail.com>
2021-06-10 21:46:53 +00:00
Yug
b63b0bf18d rbd: retrieve parent pool name of child image
when clones are created in different pool,we
need to retrieve the parent pool to get the
information of the parent image.

Co-authored-by: Madhu Rajanna <madhupr007@gmail.com>
Signed-off-by: Yug <yuggupta27@gmail.com>
2021-06-10 21:46:53 +00:00
Yug
e699318acc rbd: pass parent volume to undoSnapshotCloning function
as we are supporting the creation of clone to a new
pool we need to pass the correct parent volume
to cleanup the snapshot on parent volume.

Co-authored-by: Madhu Rajanna <madhupr007@gmail.com>
Signed-off-by: Yug <yuggupta27@gmail.com>
2021-06-10 21:46:53 +00:00
Yug
961c1d12fd rbd: add support to create clone in different pool
added support to create image in different pool.
if the snapshot/rbd image exists in one pool we
can create a clone the clone of the rbd image to
a different pool.

Co-authored-by: Madhu Rajanna <madhupr007@gmail.com>
Signed-off-by: Yug <yuggupta27@gmail.com>
2021-06-10 21:46:53 +00:00
Mohammed Naser
671d6a7767 rbd: Backout if image features is empty
In golang world, if you split an empty string that does not contain
the seperator, you get an array with one empty string.  This results
in volumes failing to mount with "invalid feature " (note extra space
because it's trying to check if 'empty string' is a valid feature).

This patch checks if the string is empty, and if so, it just decides
to skip the entire validation and returning nothing.

Signed-off-by: Mohammed Naser <mnaser@vexxhost.com>
2021-06-10 15:43:09 +00:00
Mohammed Naser
f193ebfbb1 rbd: Add failing test when no features are provided
Signed-off-by: Mohammed Naser <mnaser@vexxhost.com>
2021-06-10 15:43:09 +00:00
Madhu Rajanna
924a5ab949 build: git config before commit
currently git commit of the helm charts to
csi-chart repo is failing with below error
```
Author identity unknown

*** Please tell me who you are.

Run

git config --global user.email "you@example.com"
git config --global user.name "Your Name"

to set your account's default identity.
Omit --global to set the identity only in this repository.
```
this commit add git config for bot user.

Note:- used ceph.io in github secrets for the bot user
email and its not a valid email id and am also
looking for suggestion if anyone as.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-06-10 16:03:40 +02:00
Madhu Rajanna
2eebf6b6e0 build: use cephcsi bot token to push helm charts
GITHUB_TOKEN is auto generated for cephcsi repo
and it cannot be used to push helm charts to
different repo. added new secret CEPH_CSI_BOT_TOKEN
to push helm charts.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-06-10 12:56:10 +05:30
Madhu Rajanna
3e9aafd730 build: enclose shell variables in {}
enclose the shell variables in `{}` deploy.sh

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-06-10 10:17:19 +05:30
Madhu Rajanna
08c8272282 build: add shellcheck source=build.env in deploy.sh
added `# shellcheck source=build.env` in deploy.sh
to fix shellcheck SC1090 failure.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-06-10 10:17:19 +05:30
Madhu Rajanna
36db988f73 ci: pushing artifacts using github actions
As Travis CI `https://travis-ci.org/` is getting
shutdown date on June 15th. Either we need to move
to new place https://www.travis-ci.com/ or we can
switch to github action to push image and the helm
charts when a PR is merged.

fixes: #1781

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-06-10 10:17:19 +05:30
Yati Padia
21a400839f cleanup: No use of variable validateEncryption
In the function validatePVCSnapshot(...), we don't need
validateEncryption variable as we are passing kms value
which can help us check the value of validateEncryption.
Hence, we can avoid using that.

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-06-08 13:00:11 +00:00
Madhu Rajanna
7b5c78ec7c rbd: fail fast in create volume for missmatch encryption
CreateVolume will fail in below cases

* If the snapshot is encrypted and requested volume
is not encrypted
* If the snapshot is not encrypted and requested
volume is encrypted

* If the parent volume is encrypted and requested volume
is not encrypted
* If the parent volume is not encrypted and requested
volume is encrypted

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-06-07 15:05:21 +00:00
Niels de Vos
538e36f7a7 ci: pass GITHUB_BASE_REF when running commitlint
GitHub Actions include a merge commit for the PR, which will defeat the
commitlint checking of all the commits inside the PR (only the merge
commit is checked).

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-06-07 18:19:23 +05:30
Madhu Rajanna
4e2c4ef704 cephfs: return internal server error
if it is an error from the IsMountPoint
function and the error is not IsNotExist return
it as a internal server error.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-06-07 07:38:48 +00:00
Madhu Rajanna
46f1ab9e99 cephfs: use IsMountPoint to check mountpoint
Currently we are relaying on the error output from
the umount command we run on the nodes when mounting
the volume but we are not checking for all the error
message to verify the volume is mounted or not.
This commits uses IsMountPoint function in util
to check the mountpoint.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-06-07 07:38:48 +00:00
Madhu Rajanna
b4dbffa316 util: return actual error from IsMountPoint
as callers are already taking care of returing
the GRPC error code return the actual error
from  the IsMountPoint function.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-06-07 07:38:48 +00:00
Madhu Rajanna
fb7dc13dfe rebase: update packages in go.mod to latest releases
updated few packages in go.mod to latest
available release.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-06-04 11:52:22 +00:00
Yati Padia
0f44c6acb7 cleanup: address wasted assign issues
At places variable is reassigned without
being used.

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-06-03 09:51:14 +00:00
YingshuoTao
bfe64d4aee cephfs: pass extra volume attributes to static PV
when using pre-provisioned volumes, pass these parameters:
- kernelMountOptions
- fuseMountOptions
- subVolumeGroup
in spec.csi.volumeAttributes in PV declaration

Signed-off-by: YingshuoTao <frigid.blues@gmail.com>
2021-06-03 04:42:59 +00:00
Niels de Vos
7cbad9305f rbd: repair thick-provisioned images on CreateVolume restart
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-06-01 14:42:12 +00:00
Niels de Vos
96a8ea3e88 cleanup: split repairExistingVolume() from CreateVolume()
Move the repairing of a volume/snapshot from CreateVolume to its own
function. This reduces the complexity of the code, and makes the
procedure easier to understand. Further enhancements to repairing an
exsiting volume can be done in the new function.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-06-01 14:42:12 +00:00
Madhu Rajanna
2e978e4211 rbd: fix typo in error message
fixed typo in error message.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-06-01 10:40:07 +00:00
Madhu Rajanna
a666d452bf cephfs: return GRPC error in NodeGetVolumeStats
in case of failure return GRPC error to the caller.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-05-31 08:17:37 +00:00
Rakshith R
81809500be e2e: wait for upgraded pods to be running during upgrade-tests
This commit calls `waitForDaemonSets` and `waitForDeploymentComplete`
after upgrading to wait for csi driver pods to be in running state
for both rbd and cephfs upgrade tests.

Signed-off-by: Rakshith R <rar@redhat.com>
2021-05-26 13:15:52 +00:00
Prasanna Kumar Kalever
6984da5096 build: ignore unparam linter false positive
Ignoring below warnings:

e2e/pod.go:207:60: `execCommandInContainer` - `cn` always receives
`"csi-rbdplugin"` (unparam)
func execCommandInContainer(f *framework.Framework, c, ns, cn string,
opt *metav1.ListOptions) (string, string, error) {
                                                           ^
e2e/pod.go:308:43: `deletePodWithLabel` - `skipNotFound` always receives
`false` (unparam)
func deletePodWithLabel(label, ns string, skipNotFound bool) error {

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-05-26 10:41:34 +00:00
Prasanna Kumar Kalever
85e1e0370a e2e: enable an old testcase as the ndb module is available
This testcase tests journaling/exclusive-lock image-features with
rbd-nbd mounter

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-05-26 10:41:34 +00:00
Prasanna Kumar Kalever
819267112e e2e: restart rbd-nbd process after nodeplugin reboot
Bringup the rbd-nbd map/attach process on the rbd node plugin and expect the
IO to continue uninterrupted.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-05-26 10:41:34 +00:00
Prasanna Kumar Kalever
7334c3b783 e2e: add ability to run command inside specified container
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-05-26 10:41:34 +00:00
Prasanna Kumar Kalever
695ec6dffe e2e: Test IO after nodeplugin reboot
This is a negative testcase to showcase as per current design
the IO will fail because of the missing mappings

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-05-26 10:41:34 +00:00
Prasanna Kumar Kalever
8bae8f8458 e2e: add a test case for rbd-nbd mounter
To validate the basic working of rbd-nbd

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2021-05-26 10:41:34 +00:00
Humble Chirammal
78211b694b build: update client-go and other kube dependencies to 1.20.6
client-go 1.20.6 has a fix for below CVE: This patch address this
via updating client-go and other dependencies.

CVE-2019-11250 : The MITRE CVE dictionary describes this issue as:

The Kubernetes client-go library logs request headers at verbosity
levels of 7 or higher. This can disclose credentials to unauthorized
users via logs or command output. Kubernetes components (such as
kube-apiserver) prior to v1.16.0, which make use of basic or bearer
token authentication, and run at high verbosity levels, are affected.

Ref# https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-11250

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-05-26 09:14:10 +00:00
Rakshith R
fa1414d98f cleanup: address ineffectual assignement linter issue
updates: #1586

Signed-off-by: Rakshith R <rar@redhat.com>
2021-05-26 07:04:32 +00:00
Rakshith R
b891e5585d cleanup: address ifshort linter issues
This commit addresses ifshort linter issues which
checks if short syntax for if-statements is possible.

updates: #1586

Signed-off-by: Rakshith R <rar@redhat.com>
2021-05-26 07:04:32 +00:00
Rakshith R
6618e2012d cleanup: remove unnecessary calling of .String() when logging
This commit removes calling of .String() when logging
since `%s`,`%v` or `%q` will call an existing .String() function
automatically.

Fixes: #2051

Signed-off-by: Rakshith R <rar@redhat.com>
2021-05-25 18:02:11 +00:00
Niels de Vos
19a4d12bec rebase: update minikube to v1.20.0
See-also: https://github.com/kubernetes/minikube/releases/tag/v1.20.0
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-05-25 16:14:13 +00:00
Rakshith R
d04bfe890f helm: fix k8s version string for csidriver crds semverCompare
Current implementation of semvercompare fails against
pre-release versions. This commit fixes it by using
the entire version string at which csidriver api became GA.

s|">=1.18"|">=1.18.0-beta.1"

Fixes: #2039

Signed-off-by: Rakshith R <rar@redhat.com>
2021-05-25 14:23:33 +00:00
Madhu Rajanna
13b85d09fd ci: add wait flag to minikube start
add wait flag to minikube start to wait
for all components to be ready by default.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-05-25 12:49:12 +00:00
Madhu Rajanna
4cc6238cc4 ci: add wait-timeout flag to minikube start
minikube waits for 6 minutes for the minikube or
host to be ready, due to resource issue sometimes
the host/minikube might take longer time to start
the cluster. Increase the timeout to 10m by default

updates #1969

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-05-25 12:49:12 +00:00
Yati Padia
774e8e4042 util: enable golang profiling
Add support for golang profiling.
Standard tools like go tool pprof and curl
work. example:
$ go tool pprof http://localhost:8080/debug/pprof/profile
$ go tool pprof http://localhost:8080/debug/pprof/heap
$ curl http://localhost:8080/debug/pprof/heap?debug=1

https://golang.org/pkg/net/http/pprof/ contains
more details about the pprof interface.

Fixes: #1699

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-05-25 10:41:22 +00:00
Humble Chirammal
9aa3520c9d build: update go version to 1.16 in go.mod
Make go version latest in the repo

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-05-25 09:03:52 +00:00
Niels de Vos
2b9f6c3598 e2e: fetch volume metrics from Kubelet
Test if metrics are available at all. The actual values are a little
difficult to validate.

BlockMode volumes support metrics since Kubernetes 1.22.

See-also: kubernetes/kubernetes#97972
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-05-25 06:41:04 +00:00
Niels de Vos
25d0a1cfc0 rbd: add support for block-devices in NodeGetVolumeStats()
The NodeGetVolumeStats procedure can now be used to fetch the capacity
of the RBD block-device. By default this is a thin-provisioned device,
which means that the capacity is not reserved in the Ceph cluster. This
makes it possible to over-provision the cluster.

In order to detect the amount of storage used by the RBD block-device
(when thin-provisioned), it is required to connect to the Ceph cluster.
Unfortunately, the NodeGetVolumeStats CSI procedure does not provide
enough parameters to connect to the Ceph cluster and fetch more details
about the RBD image.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-05-25 06:41:04 +00:00
Niels de Vos
c0ab4c03e6 cephfs: move NodeGetVolumeStats() to CephFS NodeServer
The CephFS NodeServer should handle the CephFS specific requests. This
is not something that the NodeServer for RBD should handle.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-05-25 06:41:04 +00:00