As Travis CI `https://travis-ci.org/` is getting
shutdown date on June 15th. Either we need to move
to new place https://www.travis-ci.com/ or we can
switch to github action to push image and the helm
charts when a PR is merged.
fixes: #1781
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit c70e46a77c2005946af1b88d333079475c5c87d0)
This is done to prevent conflicts with current ci setup externally
attaching disks.
Signed-off-by: Rakshith R <rar@redhat.com>
(cherry picked from commit 8f75a24cfd17d9f098cd18ccb12630d33523e370)
Having double quotes around $DISK_CONFIG led to these args
not being properly passed to minikube start. This commit fixes it.
Signed-off-by: Rakshith R <rar@redhat.com>
(cherry picked from commit 42a6c3c006efc53aea57e07dd74ee2770a9128f8)
# Conflicts:
# scripts/minikube.sh
This commit disables mon,mgr and mds liveness probe
which on failing caused `crashLoopBackOff` state.
Updates: #2094
Signed-off-by: Rakshith R <rar@redhat.com>
(cherry picked from commit a4e4750fdcb0d666a29b462a34f7082a824006f6)
Rook v1.6.2 is available and this patch updates the version to the
same:
https://github.com/rook/rook/releases/tag/v1.6.2
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
(cherry picked from commit d56978739f1ff7e948f6ef21378e99f74bd1d8b4)
pylint started to report errors like the following:
troubleshooting/tools/tracevol.py:97:10: R1732: Consider using 'with' for resource-allocating operations (consider-using-with)
There probably has been an update of Pylint in the test-container that
is more strict than previous versions.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
(cherry picked from commit 8447a1feab344374c967f53de75ce88647616701)
This ensures the kubectl call is retried with kubectl_retry function.
Updates: #2309
Signed-off-by: Rakshith R <rar@redhat.com>
(cherry picked from commit 7fba62dd47d7573d2840c7df8ee38d13b7d7e21c)
It seems that the version of the key/value engine can not always be
detected for Hashicorp Vault. In certain cases, it is required to
configure the `VAULT_BACKEND` (or `vaultBackend`) option so that a
successful connection to the service can be made.
The `kv-v2` is the current default for development deployments of
Hashicorp Vault (what we use for automated testing). Production
deployments default to version 1 for now.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
(cherry picked from commit 82557e3f34ad95f4239753ad07cf9db868dad852)
Testing encrypted PVCs does not work anymore since Kubernetes v1.21. It
seems that disabling the iss validation in Hashicorp Vault is a
relatively simple workaround that we can use instead of the more complex
securing of the environment like should be done in production
deployments.
Updates: #1963
See-also: external-secrets/kubernetes-external-secrets#721
Signed-off-by: Niels de Vos <ndevos@redhat.com>
(cherry picked from commit fd9fee74dec48042c25d6934e5c166ea0e07e253)
While cleaning up snapshots, not all object may exist after a partial
provisioning attempt. In case objects are missing, do not try to delete
them.
Fixes: #2192
Signed-off-by: Niels de Vos <ndevos@redhat.com>
(cherry picked from commit 0ee0c1202732f0cc88bb1b2d799623cebbad4a35)
Getting rid of function keyword for two reasons:
1. Defining a function without 'function' keyword is more
portable as it is compatible with Bourne/Korn/POSIX scripts
2. To ensure the coding style is same for the file.
Signed-off-by: Yug <yuggupta27@gmail.com>
(cherry picked from commit e47738fa75093db3b2b2e5a04e4184bacee63e74)
The current approach uses hard-coded command line
arguments which is not very robust;
To maintain backward compatibility, script will
also keep working as the previous approach.
Signed-off-by: Yug <yuggupta27@gmail.com>
(cherry picked from commit cc72de4b1c906cde1875519141926c18d6d3de4a)
add an e2eArg `helmTest` to specify if tests are running
on ceph-csi deployment via helm.
For testing in CI, Storageclass and secret deployment
is enabled on helm installation.
Signed-off-by: Yug <yuggupta27@gmail.com>
(cherry picked from commit a4548c39832950b4523b7f4c35cecc2cf720f188)
the parent volume(CreateVolume) and the clone volume
(CreateSnapshot) are both indepedent and parent volume
can be deleted anytime. To check the thick provision
during Snapshot restore(CreateVolume from snapshot)
we need the thick provision metadata so for the same
reason setting the thick provision metadata on the
clone image we are creating at the CreateSnapshot time.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit 591ba3f580c099f4e74a7f2d39a23bf78083bbf0)
RbdSnapName holds the actual RBD image name which
got created during the CreateSnapshot operation.
RbdImageName holds the name of the parent from
which the snapshot is created. and the parent
is independent of snapshot and it can be deleted
any time for the same reason using the RbdSnapName
to check the rbd image details.
generate a temporary volume from the snapshot which
replaces the rbdImageName with RbdSnapName and use
it to check the image metadata.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit 6d14eeee70b7fb99e23723fe5164cb414d4caff6)
added validation to allow only Restore of Thick PVC
snapshot to a thick clone and creation of thick clone
from thick PVC.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit 7966d2e5c127b2d716732fa7cacf2b90d5921e9a)
isThickProvisioned can be used for both snapshot
and clone validation if isThickProvisioned is method
of common rbdImage structure.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit fc442221e4ff52d1f4f840e50bf277abf3c194be)
actual error will be present in the stdErr not the error
when we try to add a task to flatten the rbd image. This
commits corrects the error checking when the image does
not have a parent.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit 05b8433b89c3583b6c921bd831af19c6de68ec88)
The variable naming for rbd mount options has been changed
to rbdMountOptions to be consistent with other variable naming schema
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
In golang world, if you split an empty string that does not contain
the seperator, you get an array with one empty string. This results
in volumes failing to mount with "invalid feature " (note extra space
because it's trying to check if 'empty string' is a valid feature).
This patch checks if the string is empty, and if so, it just decides
to skip the entire validation and returning nothing.
Signed-off-by: Mohammed Naser <mnaser@vexxhost.com>
(cherry picked from commit 671d6a77679771bca714c7a2efe2d6799acd7fb8)
CreateVolume will fail in below cases
* If the snapshot is encrypted and requested volume
is not encrypted
* If the snapshot is not encrypted and requested
volume is encrypted
* If the parent volume is encrypted and requested volume
is not encrypted
* If the parent volume is not encrypted and requested
volume is encrypted
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit 7b5c78ec7c36b98433d19956efba2bf061dc3428)
Move the repairing of a volume/snapshot from CreateVolume to its own
function. This reduces the complexity of the code, and makes the
procedure easier to understand. Further enhancements to repairing an
exsiting volume can be done in the new function.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
(cherry picked from commit 96a8ea3e88c24e7835166f6e3c1183173fae9838)
log only the required details of
the image.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit 0ce6ad11524b3241e68057a7fb6f9aa7951e70ce)
Added an E2E test to test below case
* Create PVC
* Create Snapshot from PVC
* Delete PVC
* Create Clone from Snapshot
* Delete Snapshot
* Mount clone to Application
* Delete Application and PVC Clone
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit fa36a4668283fe3bb836c8fb4587ab32c666d898)
flatten the image if the deep-flatten feature
is present on the images in the chain or if the
images in chain is not zero, as we cannot check
the deep-flatten feature the images which are
in trash.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit 67d73cd6e9c9cede7d89667165fde4020c77a771)
For flatten we call checkImageChainHasFeature
which internally calls to getImageInfo returns
the parent name even if the parent is in the trash,
when we try to open the parent image to get its
information it fails as the image not found.
we should treat error as nil if the parent is not found.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit e15e2e5081975072fca6a6f4f7116b111eaaf308)
To recover from split brain (up+error) state the image need to be
demoted and requested for resync on site-a and then the image on site-b
should gets demoted.The volume should be marked to ready=true when the
image state on both the clusters are up+unknown because during the last
snapshot syncing the data gets copied first and then image state on the
site-a changes to up+unknown.
If the image state on both the sites are up+unknown consider that
complete data is synced as the last snapshot
gets exchanged between the clusters.
* create 10 GB of file and validate the data after resync
* Do Failover when the site-a goes down
* Force promote the image and write data in GiB
* Once the site-a comes back, Demote the image and issue resync
* Demote the image on site-b
* The status will get reflected on the other site when the last
snapshot sync happens
* The image will go to up+unknown state. and complete data will
be copied to site a
* Promote the image on site-a and use it
```bash
csi-vol-5633715e-a7eb-11eb-bebb-0242ac110006:
global_id: e7f9ec55-06ab-46cb-a1ae-784be75ed96d
state: up+unknown
description: remote image demoted
service: a on minicluster1
last_update: 2021-04-28 07:11:56
peer_sites:
name: e47e29f4-96e8-44ed-b6c6-edf15c5a91d6-rook-ceph
state: up+unknown
description: remote image demoted
last_update: 2021-04-28 07:11:41
```
* Do Failover when the site-a goes down
* Force promote the image on site-b and write data in GiB
* Demote the image on site-b
* Once the site-a comes back, Demote the image on site-a
* The images on the both site will go to split brain state
```bash
csi-vol-37effcb5-a7f1-11eb-bebb-0242ac110006:
global_id: 115c3df9-3d4f-4c04-93a7-531b82155ddf
state: up+error
description: split-brain
service: a on minicluster2
last_update: 2021-04-28 07:25:41
peer_sites:
name: abbda0f0-0117-4425-8cb2-deb4c853da47-rook-ceph
state: up+error
description: split-brain
last_update: 2021-04-28 07:25:26
```
* Issue resync
* The images cannot be resynced because when we issue resync
on site a the image on site-b was in demoted state
* To recover from this state (promote and then demote the
image on site-b after sometime)
```bash
csi-vol-37effcb5-a7f1-11eb-bebb-0242ac110006:
global_id: 115c3df9-3d4f-4c04-93a7-531b82155ddf
state: up+unknown
description: remote image demoted
service: a on minicluster1
last_update: 2021-04-28 07:32:56
peer_sites:
name: e47e29f4-96e8-44ed-b6c6-edf15c5a91d6-rook-ceph
state: up+unknown
description: remote image demoted
last_update: 2021-04-28 07:32:41
```
* Once the data is copied we can see that the image state
is moved to up+unknown on both sites
* Promote the image on site-a and use it
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit 07a916b84d8cb4e9b2f4e0399f62692d985e7a89)
when a Snapshot is encrypted during a CreateSnapshot
operation, the encryption key gets created in the KMS
when we delete the Snapshot the key from the KMS
should also gets deleted.
When we create a volume from snapshot we are copying
required information but we missed to copy the
encryption information, This commit adds the missing
information to delete the encryption key.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit c3bae17fcee0413dfb0a15a3524f0e7e0fc8d5d9)
This reverts commit a07260f19153cb6fef7cb27bfb9135082630830e.
which had template changes for v3.3.1 release.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
At present we return the volume connect error if the clone
from snapshot fails when rbdvolume is encrypted, which is incorrect.
This patch correctly return the failed copy encryption error to the
caller
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
(cherry picked from commit 798437d0c4cc970187f65ec110684b597f98cf49)
from helm v3.x version there is no helm init
command. Removing the helm init which was causing
helm chart pushing issue in release and devel
branch.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit 6508726276748c5618ee1972bf14f018135be39c)
Install the helm package based on the version
specified in the build.env
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit aa77b677a32dfc5bc4a2048da10f741409b2dcb5)
Before RBD map operation, we do check the
watchers on the RBD image. In the case of
RWO volume. cephcsi makes sure only one
client is using the RBD image. If the rbd
image is mirrored, by default mirroring
daemon will add a watcher on the image
and as we are using go-ceph a watcher will
be added as we have opened the image So
we will have two watchers on an image if
mirroring is enabled. This holds when the
rbd mirror daemon is running, In case if
the mirror daemon is not running there will
be only one watcher on the rbd image
(which is placed by go-ceph image open)
we should not block the map operation if
the mirroring daemon is not running as
its Async mirroring. This commit adds a
check to make sure no more than 2 watchers
if the image is mirrored or no more than 1
watcher if it is not mirrored image.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit 52290333e61bf5e659ab8ff22c412366b42db126)
This commit reverts back the changes done
for v3.3.0 release. With this change a
release canary tagged image and helm charts
will get pushed.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
If the pool or few keys are missing in the omap.
GetImageAttributes function returns nil error message and few
empty items in imageAttributes struct. if the image is not
found and the entiries are missing use
the volumeId present on the PV annotation for further operations.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
