The jjb-validate job can not be run in parallel, so it may fail when
multiple PRs for the ci/centos branch are sent. However, the ci/centos
branch is not very active, so problems should be minimal.
In case of problems, leave the following comment in the PR and the job
should restart:
/retest ci/centos/jjb-validate
See-also: #1273
Signed-off-by: Niels de Vos <ndevos@redhat.com>
While adding the context.Context to the resizeRBDimage() function, it
became a little ugly. So renaming the function to resize() and making it
a method of the rbdVolume type.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Most consumers of util.ExecCommand() need to convert the returned []byte
format of stdout and/or stderr to string. By having util.ExecCommand()
return strings instead, the code gets a little simpler.
A few commands return JSON that needs to be parsed. These commands will
be replaced by go-ceph implementations later on. For now, convert the
strings back to []byte when needed.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
All calls to util.ExecCommand() now pass the context.Context. In some
cases this is not possible or needed, and util.ExecCommand() will not
log the command.
This should make debugging easier when command executions fail.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Current versions of the mkfs.xfs binary enable reflink support by
default. This causes problems on systems where the kernel does not
support this feature. When the kernel the feature does not support, but
the filesystem has it enabled, the following error is logged in `dmesg`:
XFS: Superblock has unknown read-only compatible features (0x4) enabled
Introduce a check to see if mkfs.xfs supports the `-m reflink=` option.
In case it does, pass `-m reflink=0` while creating the filesystem.
The check is executed once during the first XFS filesystem creation. The
result of the check is cached until the nodeserver restarts.
Fixes: #966
Signed-off-by: Niels de Vos <ndevos@redhat.com>
The new rbdVolume.isInUse() method will replace the rbdStatus()
function. This removes one more rbd command execution in the
DeleteVolume path.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
As part of https://github.com/ceph/ceph-csi/pull/1237/ there was
a patching enabled for the ceph cluster deployed, however due to
an error in the version fetching logic, the patching was not applied
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
This change replaces the sentinel errors in rbd module with
standard errors created with errors.New().
Related: #1203
Signed-off-by: Sven Anderson <sven@redhat.com>
This change replaces the sentinel errors in cephfs module with
standard errors created with errors.New().
Related: #1203
Signed-off-by: Sven Anderson <sven@redhat.com>
The sentinel error code had additional fields in the errors, that are
used nowhere. This leads to unneccesarily complicated code. This
change replaces the sentinel errors in utils with standard errors
created with errors.New() and adds a simple JoinErrors() function to
be able to combine sentinel errors from different code tiers.
Related: #1203
Signed-off-by: Sven Anderson <sven@redhat.com>
In rbd E2E testing,we need to create snap and clone
as parallel operation.
This helps us to insure that functionality works when
we have parallel delete and create operations and also
it helps to catch bugs when we get parallel requests.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
Take operation locks on the resources before operating
on the resouces. This allows us to do parallel operations
for some RPC calls such as Clone and Restore of PVC.
This operations will only be blocked if the image is
expanding or Snapshot and RBD image is getting deleted.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
As we are adding new functionalities like Create/Delete
snapshot,Clone from Snapshot and Clone from Volume.
with the current implementation, there are only serial
operations allowed for this functionalities, for some
function we can allow parallel operations like
Clone from snapshot and Clone from Volume and Create
`N` snapshots on a single volume.
Delete Volume: Need to ensure that there is no clone,
Snapshot create and Expand volume in progress.
Expand Volume: Need to ensure that there is no clone,
snapshot create and cloning in progress
Delete Snapshot: Need to ensure that there is no
cloning in progress
Restore Volume/Snapshot: Need to ensure that there is
no Expand or delete operation in progress.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
Direct usage of numbers should be avoided.
Issue reported:
mnd: Magic number: X, in <argument> detected (gomnd)
Signed-off-by: Yug <yuggupta27@gmail.com>
Add explanation to nolint directives.
Issue reported:
whyNoLint: include an explanation for nolint directive (gocritic)
Signed-off-by: Yug <yuggupta27@gmail.com>
In Go 1.13, the fmt.Errorf function supports a new %w verb.
When this verb is present, the error returned by fmt.Errorf
will have an Unwrap method returning the argument of %w,
which must be an error. In all other ways, %w is identical to %v.
Updates: #1227
Signed-off-by: Yug <yuggupta27@gmail.com>
We had "ns" as a parameter and then trying to
declare it also as a local variable, which is what
the complaint about "shadowing" refers to.
Issue reported:
shadow: declaration of "ns" shadows declaration at line 57 (govet)
Signed-off-by: Yug <yuggupta27@gmail.com>
goerr113 linter checks the errors handling expressions.
It warns about using wrapped static errors in place
of dynamic at multiple places.
Disabled the linter as of now to avoid regression,
and this need to be handled in a seperate issue.
Tracker Issue: #1227
Signed-off-by: Yug <yuggupta27@gmail.com>
The `nestif` linter reports deeply nested if statements.
Disabled `nestif` as of now to avoid regression and
needs to addressed seperately.
Tracking Issue: #1229
Signed-off-by: Yug <yuggupta27@gmail.com>
The 'testpackage' linter recommends "black box" testing, which prevents
testing internal/non-exported functions from being tested. We have tests
that *do* test non-exported functions and types.
Disabling the linter allows us to test non-exported types and functions.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Rook version is currently 1.1.7 in our e2e deployment which brings 14.2.4 version
of ceph cluster. To support cephfs snapshot e2e, we need latest version of Ceph Cluster
in E2E. Rook 1.2.7 is good enough which on patching bring up ceph 14.2.10 cluster.
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
Added a document which contains the steps
and RBD CLI commands we execute when we create
a kubernetes snapshot, delete kubernetes snapshot,
Restore a snapshot to a new PVC,Kubernetes volume
cloning and kubernetes PVC deletion.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
Since containerized-tests job is functioning
again, we can re-enable the job as a required
condition to merge PRs.
Signed-off-by: Yug <yuggupta27@gmail.com>
In some ceph version if the subvolume is not present, the
ceph returns doesnot exists and in some version not found
error message. This commit fixes issue for both error
checks.
By only checking Error ENOENT: for doesnot exist seems good.
even if some error message changes in ceph ceph-csi wont get
any issue.
```bash
sh-4.2# ceph version
ceph version 14.2.10 (b340acf629a010a74d90da5782a2c5fe0b54ac20) nautilus (stable)
sh-4.2# ceph fs subvolume getpath myfs csi-vol-a24a3d97-c7f4-11ea-8cfc-0242ac110012 --group_name csi
Error ENOENT: subvolume 'csi-vol-a24a3d97-c7f4-11ea-8cfc-0242ac110012' does not exist
```
```bash
sh-4.2# ceph version
ceph version 14.2.4 (75f4de193b3ea58512f204623e6c5a16e6c1e1ba) nautilus (stable)
sh-4.2# ceph fs subvolume getpath myfs testing --group_name=csi
Error ENOENT: Subvolume 'testing' not found
```
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
This is a workaround to fix docker permission denied issue
during manifest create in Travis CI
`docker manifest create` fails due to permission denied
on `/etc/docker/certs.d/quay.io`
more info https://github.com/docker/for-linux/issues/396.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
If the requested volume size and the snapshot or the
parent volume from which the clone is to be created
is not equal cephcsi returns an error message.
updates #1188
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
As quay.io doesnot support the multi architecture
images, We need to switch to dockerhub as it supports
multi-architecture images.
closes: #1003
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
If the image in the chain is moved to trash, we
cannot get the image details. We need to return the
found depth to the caller.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
If the snapshots on the parent image exceeds
maxSnapshotsOnImage count, we need to flatten
all the temporary cloned images to over come the
krbd issue of maximum number of snapshots on
an image.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
create temporary snapshot on the parent image same as
name as the temporary clone rbd image. Naming the snapshot
and the temporary cloned image helps to flatten the temporary
cloned images when the snapshots on the parent image exceeds
the configured maxSnapshotsOnImage.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
CentOS CI outage block PRs to get merged even when it has enough
approvals. Disabling the centos check for now as there is no ETA
on the availability of the setup atm.
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
rename newVolumeOptionsFromVersion1Context to newVolumeOptionsFromMonitorList
to provide more clarity to the function readers and also fixed comments.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
as v1.0.0 is deprecated we need to remove the support
for it in the Next coming (v3.0.0) release. This PR
removes the support for the same.
closes#882
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
Added support for RBD PVC to PVC cloning, below
commands are executed to create a PVC-PVC clone from
RBD side.
* Check the depth(n) of the cloned image if n>=(hard limit -2)
or ((soft limit-2) Add a task to flatten the image and return
about (to avoid image leak) **Note** will try to flatten the
temp clone image in the chain if available
* Reserve the key and values in omap (this will help us to
avoid the leak as it's not reserved earlier as we have returned
ABORT (the request may not come back))
* Create a snapshot of rbd image
* Clone the snapshot (temp clone)
* Delete the snapshot
* Snapshot the temp clone
* Clone the snapshot (final clone)
* Delete the snapshot
```bash
1) check the image depth of the parent image if flatten required
add a task to flatten image and return ABORT to avoid leak
(hardlimit-2 and softlimit-2 check will be done)
2) Reserve omap keys
2) rbd snap create <RBD image for src k8s volume>@<random snap name>
3) rbd clone --rbd-default-clone-format 2 --image-feature
layering,deep-flatten <RBD image for src k8s volume>@<random snap>
<RBD image for temporary snap image>
4) rbd snap rm <RBD image for src k8s volume>@<random snap name>
5) rbd snap create <cloned RBD image created in snapshot process>@<random snap name>
6) rbd clone --rbd-default-clone-format 2 --image-feature <k8s dst vol config>
<RBD image for temporary snap image>@<random snap name> <RBD image for k8s dst vol>
7)rbd snap rm <RBD image for src k8s volume>@<random snap name>
```
* Delete temporary clone image created as part of clone(delete if present)
* Delete rbd image
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
kubernetes v1.17.8 is released recently,This commit
updates the kubernetes v1.17 to latest patch release
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
