Rook version is currently 1.1.7 in our e2e deployment which brings 14.2.4 version
of ceph cluster. To support cephfs snapshot e2e, we need latest version of Ceph Cluster
in E2E. Rook 1.2.7 is good enough which on patching bring up ceph 14.2.10 cluster.
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
Added a document which contains the steps
and RBD CLI commands we execute when we create
a kubernetes snapshot, delete kubernetes snapshot,
Restore a snapshot to a new PVC,Kubernetes volume
cloning and kubernetes PVC deletion.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
Since containerized-tests job is functioning
again, we can re-enable the job as a required
condition to merge PRs.
Signed-off-by: Yug <yuggupta27@gmail.com>
In some ceph version if the subvolume is not present, the
ceph returns doesnot exists and in some version not found
error message. This commit fixes issue for both error
checks.
By only checking Error ENOENT: for doesnot exist seems good.
even if some error message changes in ceph ceph-csi wont get
any issue.
```bash
sh-4.2# ceph version
ceph version 14.2.10 (b340acf629a010a74d90da5782a2c5fe0b54ac20) nautilus (stable)
sh-4.2# ceph fs subvolume getpath myfs csi-vol-a24a3d97-c7f4-11ea-8cfc-0242ac110012 --group_name csi
Error ENOENT: subvolume 'csi-vol-a24a3d97-c7f4-11ea-8cfc-0242ac110012' does not exist
```
```bash
sh-4.2# ceph version
ceph version 14.2.4 (75f4de193b3ea58512f204623e6c5a16e6c1e1ba) nautilus (stable)
sh-4.2# ceph fs subvolume getpath myfs testing --group_name=csi
Error ENOENT: Subvolume 'testing' not found
```
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
This is a workaround to fix docker permission denied issue
during manifest create in Travis CI
`docker manifest create` fails due to permission denied
on `/etc/docker/certs.d/quay.io`
more info https://github.com/docker/for-linux/issues/396.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
If the requested volume size and the snapshot or the
parent volume from which the clone is to be created
is not equal cephcsi returns an error message.
updates #1188
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
As quay.io doesnot support the multi architecture
images, We need to switch to dockerhub as it supports
multi-architecture images.
closes: #1003
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
If the image in the chain is moved to trash, we
cannot get the image details. We need to return the
found depth to the caller.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
If the snapshots on the parent image exceeds
maxSnapshotsOnImage count, we need to flatten
all the temporary cloned images to over come the
krbd issue of maximum number of snapshots on
an image.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
create temporary snapshot on the parent image same as
name as the temporary clone rbd image. Naming the snapshot
and the temporary cloned image helps to flatten the temporary
cloned images when the snapshots on the parent image exceeds
the configured maxSnapshotsOnImage.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
CentOS CI outage block PRs to get merged even when it has enough
approvals. Disabling the centos check for now as there is no ETA
on the availability of the setup atm.
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
rename newVolumeOptionsFromVersion1Context to newVolumeOptionsFromMonitorList
to provide more clarity to the function readers and also fixed comments.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
as v1.0.0 is deprecated we need to remove the support
for it in the Next coming (v3.0.0) release. This PR
removes the support for the same.
closes#882
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
Added support for RBD PVC to PVC cloning, below
commands are executed to create a PVC-PVC clone from
RBD side.
* Check the depth(n) of the cloned image if n>=(hard limit -2)
or ((soft limit-2) Add a task to flatten the image and return
about (to avoid image leak) **Note** will try to flatten the
temp clone image in the chain if available
* Reserve the key and values in omap (this will help us to
avoid the leak as it's not reserved earlier as we have returned
ABORT (the request may not come back))
* Create a snapshot of rbd image
* Clone the snapshot (temp clone)
* Delete the snapshot
* Snapshot the temp clone
* Clone the snapshot (final clone)
* Delete the snapshot
```bash
1) check the image depth of the parent image if flatten required
add a task to flatten image and return ABORT to avoid leak
(hardlimit-2 and softlimit-2 check will be done)
2) Reserve omap keys
2) rbd snap create <RBD image for src k8s volume>@<random snap name>
3) rbd clone --rbd-default-clone-format 2 --image-feature
layering,deep-flatten <RBD image for src k8s volume>@<random snap>
<RBD image for temporary snap image>
4) rbd snap rm <RBD image for src k8s volume>@<random snap name>
5) rbd snap create <cloned RBD image created in snapshot process>@<random snap name>
6) rbd clone --rbd-default-clone-format 2 --image-feature <k8s dst vol config>
<RBD image for temporary snap image>@<random snap name> <RBD image for k8s dst vol>
7)rbd snap rm <RBD image for src k8s volume>@<random snap name>
```
* Delete temporary clone image created as part of clone(delete if present)
* Delete rbd image
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
kubernetes v1.17.8 is released recently,This commit
updates the kubernetes v1.17 to latest patch release
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
As kube v1.18 is released we dont need to test
-2 version in our CI, removing the 1.16 testing
for the same reason.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
With minikube versions greater than 1.6.2 and less than 1.11.1, the YAML files
minikube path will not be automatically applied to the cluster. we will get
errors during bootstrap of the cluster if the admission controller is enabled.
To use Pod Security Policies with these versions of minikube, first start a
cluster without the `PodSecurityPolicy` admission controller enabled.
Next, apply the psp yaml. and stop the cluster and then restart it
with the admission controller enabled.
```
minikube start
kubectl apply -f /path/to/psp.yaml
minikube stop
minikube start --extra-config=apiserver.enable-admission-plugins=PodSecurityPolicy
```
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
As the maximum time allocated for the Travis
CI is 50 minutes and the cephfs and rbd E2E
takes around 45 Minutes to run completely.
This is blocking us from adding more tests in E2E.
splitting out the E2E will help us to run more tests
for each driver as we will get 50 minutes for
each Travis CI instance.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
Added two new parameters for e2e test to skip
rbd and cephfs tests. This will help us to
run more test in Travis CI.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
during the checkSnapCloneExists we are checking
the image, if the image not found we are deleting
the snapshot on the parent image, This PR corrects
the comparasion. instead of snapshotNotFound we need
to check ImageNotFound error.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
tarcevol.py works with a fixed volume name prefix "csi-vol-", this could be changed by the storageclass
and hence tracevol.py should work with the dynamic value rather than the static.
Signed-off-by: Mudit Agarwal <muagarwa@redhat.com>
updated unit testing for the kernel check
for deep flatten feature for both supported
upstream kernel version (5.1.0+) and RHEL
8.2 backport
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
In case Podman or Docker is not installed, running containerized tests
or builds will fail with unclear errors.
Introduce a 'need-container-cmd' make target that checks if the
CONTAINER_CMD is available (and non-empty). When running without Podman
and Docker, the following error is displayed:
$ make containerized-build
cephcsi image settings: quay.io/cephcsi/cephcsi version canary
Missing container support, install Podman or Docker
make: *** [Makefile:135: need-container-cmd] Error 1
Signed-off-by: Niels de Vos <ndevos@redhat.com>
as RHEL 8.2 supports the deep-flatten
feature, added it to the list to map
the rbd images on the node without flattening.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
as v5.1.0 supports the deep-flatten feature,lowering
the required version to map rbd images which
are having deep-flatten feature
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
Reduced the number of pods created
in ROX E2E to save some time in E2E
and changed the waiting time from 2 to 1
min.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
as we are creating more resources
the e2e timeout is not enough we need
to increase the e2e timeout to higher
value for now.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
we need to take lock on parent rbd image when
we are creating a snapshot from it, if the user
tries to delete/resize the rbd image when we are
taking snapshots,we may face issues. if the volume
lock is present on the rbd image, the user cannot
resize the rbd image nor delete the rbd image.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
as we need to test the maxsnapshotsonimage we
need to set the limit to minimal value which we
can test in CI as the default limit is 450,which
cannot be tested in CI.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
flatten cloned images to remove the link
with the snapshot as the parent snapshot can
be removed from the trash.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
Added maxsnapshotsonimage flag to flatten
the older rbd images on the chain to avoid
issue in krbd.The limit is in krbd since it
only allocate 1 4KiB page to handle all the
snapshot ids for an image.
The max limit is 510 as per
https://github.com/torvalds/linux/blob/
aaa2faab4ed8e5fe0111e04d6e168c028fe2987f/drivers/block/rbd.c#L98
in cephcsi we arekeeping the default to 450 to reserve 10%
to avoid issues.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
