Commit Graph

2089 Commits

Author SHA1 Message Date
Madhu Rajanna
6d14eeee70 rbd: use RbdSnapName to check the image details
RbdSnapName holds the actual RBD image name which
got created during the CreateSnapshot operation.
RbdImageName holds the name of the parent from
which the snapshot is created. and the parent
is independent of snapshot and it can be deleted
any time for the same reason using the RbdSnapName
to check the rbd image details.

generate a temporary volume from the snapshot which
replaces the rbdImageName with RbdSnapName and use
it to check the image metadata.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-06-18 10:57:48 +00:00
Madhu Rajanna
7966d2e5c1 rbd: add validation for thick restore/clone
added validation to allow only Restore of Thick PVC
snapshot to a thick clone and creation of thick clone
from thick PVC.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-06-18 10:57:48 +00:00
Madhu Rajanna
fc442221e4 rbd: make isThickProvisioned method of rbdImage
isThickProvisioned can be used for both snapshot
and clone validation if isThickProvisioned is method
of common rbdImage structure.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-06-18 10:57:48 +00:00
Niels de Vos
57d3183cb1 rbd: restart thick-provisioned PVC cloning after aborting
In case cloning a thick-PVC failed during DeepCopy(), the image will
exist, but have partial contents. Only when the image has the
thick-provisioned metadata set, it has completed DeepCopy().

When the metadata is missing, the image is deleted, and an error is
returned to the caller. Kubernetes will automatically retry provisioning
on the ABORTED error, and the cloning will get restarted from the
beginning.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-06-18 06:25:56 +00:00
Niels de Vos
b1045364d9 rbd: disable FeatureDeepFlatten when doing DeepCopy()
Not all Linux kernels support the deep-flatten feature. Disabling the
feature makes it possible to map RBD images on older kernels (like what
minikube uses).

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-06-18 06:25:56 +00:00
Niels de Vos
4908ff8743 rbd: no need to flatten thick-provisioned images
Thick-provisioned images are independent, cloned images or snapshots are
deep-flattened during creation. There is no need to try and flatten them
again.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-06-18 06:25:56 +00:00
Niels de Vos
0fe0962dc1 e2e: validate thick-provisioned PVC-PVC cloning
Add a case to create a new PVC with VolumeContentSource set to a
thick-provisioned PVC. This should result in a new thick-provisioned PVC
once the cloning is done.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-06-18 06:25:56 +00:00
Niels de Vos
6cc11c15d3 rbd: use DeepCopy to create a thick-provisioned clone
To create a full-allocated RBD image from a snapshot/clone DeepCopy()
can be used. This is needed when the parent of the new volume is
thick-provisioner, so that the new volume is independent of the parent
and thick-provisioned as well.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-06-18 06:25:56 +00:00
Niels de Vos
334f237e23 cleanup: move snapshot/clone/flatten into its own function
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-06-18 06:25:56 +00:00
Madhu Rajanna
367eb9f748 rbd: correct return error for isCompatibleEncryption
isCompatibleEncryption is used to validate the
requested volume and the existing volume and
the destination volume name wont be generated yet
and logging the destination volume prints the empty
image name with pool name.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-06-17 10:12:18 +00:00
Madhu Rajanna
9fde4f7f18 ci: disable push artifacts on fork repos
disabling push artifacts github action on the
fork repos as it doesnot makes sense to run push
actions on the fork repo.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-06-17 10:11:13 +02:00
Niels de Vos
f66b21873b e2e: pass --delete-on-failure when starting minikube
Sometimes there is an unclear error while starting the Minikube VM.
Possibly this can be worked around by passing the --delete-on-failure
option:

    If set, delete the current cluster if start fails and try again. Defaults to false.

See-also: https://minikube.sigs.k8s.io/docs/commands/start/
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-06-16 10:59:17 +00:00
Rakshith R
ba4e92eea9 doc: add imageFeatures parameter info to static-pvc.md
Signed-off-by: Rakshith R <rar@redhat.com>
2021-06-16 06:56:15 +00:00
Madhu Rajanna
c6bc84d847 e2e: validate images in trash
when all the PVC and associated images are deleted,
the images should also get deleted from the trash.
This commit adds the validation check for the same.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-06-15 15:38:24 +00:00
Madhu Rajanna
05b8433b89 rbd: check stdErr for does not have a parent error
actual error will be present in the stdErr not the error
when we try to add a task to flatten the rbd image. This
commits corrects the error checking when the image does
not have a parent.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-06-15 11:07:34 +00:00
Niels de Vos
eeb2ad8fac rebase: update minikube to v1.21.0
See-also: https://github.com/kubernetes/minikube/releases/tag/v1.21.0
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-06-15 08:20:57 +00:00
Yati Padia
cfdf5af08f doc: adds note to replace canary with version tag
This commit adds a note to replace canary with version
tag when deploying in production.

Fixes: #2156

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-06-15 03:14:44 +00:00
Yati Padia
6bfdf2feb0 cleanup: gocyclo being unused for linter
This commit addresses the following issue:
'nolint:gocyclo // complexity needs to be reduced.'
is unused for linter "gocyclo" (nolintlint)

Updates:#2025

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-06-15 02:54:16 +00:00
Yug
5c079894c7 doc: correct comment indentation in rbdVolume
correct comment indentation in rbdvolume{}

Signed-off-by: Yug <yuggupta27@gmail.com>
2021-06-15 02:34:51 +00:00
Yati Padia
095a82f37d util: returns actual error instead of ErrPoolNotFound
This commit returns actual error returned by the go-ceph API
to the function GetPoolName(..) instead of just returning
ErrPoolNotFound everytime there is error getting the pool id.
There is a issue reported in which the snapshot creation
takes much more time to reach True state
(i.e., between 2-7 mins) and keeps trying to create with
below error though pool is present:
rpc error: code = NotFound desc = pool not found: pool ID (21)
not found in Ceph cluster.

Since we cannot interpret the actual error for the delay in
snapshot creation, it is required to return the actual error
as well so that we can uderstand the reason.

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-06-14 14:41:32 +00:00
Madhu Rajanna
5b7b5f1e3a rebase: update go-ceph to v0.10.0
This commit updates the go-ceph to latest
release. More details about release at
https://github.com/ceph/go-ceph/releases/tag/v0.10.0

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-06-11 09:36:53 +00:00
Humble Chirammal
17b0091cba cleanup: fix codespell error in internal/utils package
Codespell checker report below error:
```
Resulting CLI options  --check-filenames --check-hidden --skip .git,./vendor --ignore-words-list ExtraVersion,extraversion,ba
1
Error: ./internal/util/aws_metadata.go:96: Kubenetes ==> Kubernetes
```
This commit address the same.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-06-11 08:04:07 +00:00
Humble Chirammal
4302e5ee67 cleanup: correct createORDeleteCephfsResources() function name
Along with correcing the name of the function, other typos are also
addressed

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-06-11 08:04:07 +00:00
Humble Chirammal
a9fa108959 e2e: use proper variable name for rbd mount options
The variable naming for rbd mount options has been changed
to rbdMountOptions to be consistent with other variable naming schema

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-06-11 08:04:07 +00:00
Humble Chirammal
5635e99e37 e2e: correct gosec marker for credentials rule
The marker for hardcoded credentials check was set wrongly
and this patch address the same

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-06-11 08:04:07 +00:00
Humble Chirammal
2851c7d17a cleanup: correct createORdeleteRbdResources() function name
This patch address a typo in the mentioned function name

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-06-11 08:04:07 +00:00
Yug
33bf3fae6b e2e: add logErr function to log errors
logErr function logs all the ocured errors
with a message that is passed for occurence
of each error.

Co-authored-by: Niels de Vos <ndevos@redhat.com>
Co-authored-by: Madhu Rajanna <madhupr007@gmail.com>
Signed-off-by: Yug <yuggupta27@gmail.com>
2021-06-10 21:46:53 +00:00
Yug
8d038db047 e2e: add e2e to test clone in different pool
added a e2e to test clones in different
pool.

Co-authored-by: Madhu Rajanna <madhupr007@gmail.com>
Signed-off-by: Yug <yuggupta27@gmail.com>
2021-06-10 21:46:53 +00:00
Yug
1b2e9f556f e2e: add helper function for clone validation
added a helper function to test clone creation
in a different pool.

Co-authored-by: Madhu Rajanna <madhupr007@gmail.com>
Signed-off-by: Yug <yuggupta27@gmail.com>
2021-06-10 21:46:53 +00:00
Yug
91662b54b6 e2e: move yaml filepath declaration to global
declared yaml filepath in gobal for reusing.

Co-authored-by: Madhu Rajanna <madhupr007@gmail.com>
Signed-off-by: Yug <yuggupta27@gmail.com>
2021-06-10 21:46:53 +00:00
Yug
30092ff4e1 e2e: add an option to set rbd storageclass name
added an option to set storageclass name.

Co-authored-by: Madhu Rajanna <madhupr007@gmail.com>
Signed-off-by: Yug <yuggupta27@gmail.com>
2021-06-10 21:46:53 +00:00
Yug
76e9b4b02a e2e: add function to create rbd pool
added a new function to create rbd pool
in e2e.

Co-authored-by: Madhu Rajanna <madhupr007@gmail.com>
Signed-off-by: Yug <yuggupta27@gmail.com>
2021-06-10 21:46:53 +00:00
Yug
e302797217 e2e: update listRBDImages to list images in different pool
made pool as a argument of listRBDImages to support
listing of rbd images in different pools.

Co-authored-by: Madhu Rajanna <madhupr007@gmail.com>
Signed-off-by: Yug <yuggupta27@gmail.com>
2021-06-10 21:46:53 +00:00
Yug
d992803e9e rbd: Update pool name in image chain
While traversing image chain, the parent
image can be present in a different pool
that the one child is in. So, updating
pool name in the next itteration to
that of the Parent.

Co-authored-by: Madhu Rajanna <madhupr007@gmail.com>
Signed-off-by: Yug <yuggupta27@gmail.com>
2021-06-10 21:46:53 +00:00
Yug
1f6a9cabfd rbd: verify if pool name is not empty
Validate Snapshot request to check if the
passed pool name is not empty.

Co-authored-by: Madhu Rajanna <madhupr007@gmail.com>
Signed-off-by: Yug <yuggupta27@gmail.com>
2021-06-10 21:46:53 +00:00
Yug
3898ae34a7 rbd: open new ioctx connection
if the parent and child clones are in
different namespaces we need to open a new
ioctx for pools.

Co-authored-by: Madhu Rajanna <madhupr007@gmail.com>
Signed-off-by: Yug <yuggupta27@gmail.com>
2021-06-10 21:46:53 +00:00
Yug
b63b0bf18d rbd: retrieve parent pool name of child image
when clones are created in different pool,we
need to retrieve the parent pool to get the
information of the parent image.

Co-authored-by: Madhu Rajanna <madhupr007@gmail.com>
Signed-off-by: Yug <yuggupta27@gmail.com>
2021-06-10 21:46:53 +00:00
Yug
e699318acc rbd: pass parent volume to undoSnapshotCloning function
as we are supporting the creation of clone to a new
pool we need to pass the correct parent volume
to cleanup the snapshot on parent volume.

Co-authored-by: Madhu Rajanna <madhupr007@gmail.com>
Signed-off-by: Yug <yuggupta27@gmail.com>
2021-06-10 21:46:53 +00:00
Yug
961c1d12fd rbd: add support to create clone in different pool
added support to create image in different pool.
if the snapshot/rbd image exists in one pool we
can create a clone the clone of the rbd image to
a different pool.

Co-authored-by: Madhu Rajanna <madhupr007@gmail.com>
Signed-off-by: Yug <yuggupta27@gmail.com>
2021-06-10 21:46:53 +00:00
Mohammed Naser
671d6a7767 rbd: Backout if image features is empty
In golang world, if you split an empty string that does not contain
the seperator, you get an array with one empty string.  This results
in volumes failing to mount with "invalid feature " (note extra space
because it's trying to check if 'empty string' is a valid feature).

This patch checks if the string is empty, and if so, it just decides
to skip the entire validation and returning nothing.

Signed-off-by: Mohammed Naser <mnaser@vexxhost.com>
2021-06-10 15:43:09 +00:00
Mohammed Naser
f193ebfbb1 rbd: Add failing test when no features are provided
Signed-off-by: Mohammed Naser <mnaser@vexxhost.com>
2021-06-10 15:43:09 +00:00
Madhu Rajanna
924a5ab949 build: git config before commit
currently git commit of the helm charts to
csi-chart repo is failing with below error
```
Author identity unknown

*** Please tell me who you are.

Run

git config --global user.email "you@example.com"
git config --global user.name "Your Name"

to set your account's default identity.
Omit --global to set the identity only in this repository.
```
this commit add git config for bot user.

Note:- used ceph.io in github secrets for the bot user
email and its not a valid email id and am also
looking for suggestion if anyone as.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-06-10 16:03:40 +02:00
Madhu Rajanna
2eebf6b6e0 build: use cephcsi bot token to push helm charts
GITHUB_TOKEN is auto generated for cephcsi repo
and it cannot be used to push helm charts to
different repo. added new secret CEPH_CSI_BOT_TOKEN
to push helm charts.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-06-10 12:56:10 +05:30
Madhu Rajanna
3e9aafd730 build: enclose shell variables in {}
enclose the shell variables in `{}` deploy.sh

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-06-10 10:17:19 +05:30
Madhu Rajanna
08c8272282 build: add shellcheck source=build.env in deploy.sh
added `# shellcheck source=build.env` in deploy.sh
to fix shellcheck SC1090 failure.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-06-10 10:17:19 +05:30
Madhu Rajanna
36db988f73 ci: pushing artifacts using github actions
As Travis CI `https://travis-ci.org/` is getting
shutdown date on June 15th. Either we need to move
to new place https://www.travis-ci.com/ or we can
switch to github action to push image and the helm
charts when a PR is merged.

fixes: #1781

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-06-10 10:17:19 +05:30
Yati Padia
21a400839f cleanup: No use of variable validateEncryption
In the function validatePVCSnapshot(...), we don't need
validateEncryption variable as we are passing kms value
which can help us check the value of validateEncryption.
Hence, we can avoid using that.

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-06-08 13:00:11 +00:00
Madhu Rajanna
7b5c78ec7c rbd: fail fast in create volume for missmatch encryption
CreateVolume will fail in below cases

* If the snapshot is encrypted and requested volume
is not encrypted
* If the snapshot is not encrypted and requested
volume is encrypted

* If the parent volume is encrypted and requested volume
is not encrypted
* If the parent volume is not encrypted and requested
volume is encrypted

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-06-07 15:05:21 +00:00
Niels de Vos
538e36f7a7 ci: pass GITHUB_BASE_REF when running commitlint
GitHub Actions include a merge commit for the PR, which will defeat the
commitlint checking of all the commits inside the PR (only the merge
commit is checked).

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-06-07 18:19:23 +05:30
Madhu Rajanna
4e2c4ef704 cephfs: return internal server error
if it is an error from the IsMountPoint
function and the error is not IsNotExist return
it as a internal server error.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-06-07 07:38:48 +00:00