If a CreateVolume call is interrupted,
post creating the required CSI journal entries,
but prior to creating the backing CephFS subvolume,
then a subsequent CreateVolume call will return
a valid response with a VolumeID that has
it's backing image missing. This PR adds a check
for backend image, if image notfound it deletes the
reserved keys in omap.
fixes#839
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit 128f3fc2cf)
Added checks in DeleteVolume RPC, for image missing errors, and
taking appropriate actions to cleanup the CSI reservations.
Further removed forcing a volume purge, and instead added checks
for missing volume errors in purgeVolume.
This should now fix issues where an continuation of an interrupted
DeleteVolume call, that only deleted the backing volume, will
proceed and not error out.
Signed-off-by: ShyamsundarR <srangana@redhat.com>
(cherry picked from commit 35e8c3b3a5)
This values will be added at both nodestage
and nodepublish for rbd, nbd and ceph kernel client.
As cephfs fuse doesnot support this value,
this is added only during the nodepublish.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit 881f59d142)
if the directory is not mounted return nil
during umount of mountPoint
Discard error if error is os.IsNotExist
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
Linux kernel 4.17.0 adds support for quota with CephFS. Without quota,
it is not possible to fullfill the requirements of the CSI Spec and
guarantee sufficient space on the filesystem for a volume. With this in
mind, usage of the kernel client is only allowed with kernel 4.17.0 or
newer.
However, some Linux vendors backport features and patches to their
Enterprise products. These kernels may have an older version, but do
support quota. One of these is the kernel that comes with RHEL-7.7.
By comparing the current running version of the Linux kernel against
known versions that support quota, we can now automatically decide to
use the kernel client, or not.
Note that this does not change the 'forcekernelclient' parameter. The
parameter is still available and can be used for kernels that are not in
the 'known to support quota list'. Or users can pass the parameter to
use a CephFS kernel client that does not support quota.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
The storage class already takes MountOptions(MountFlags), these are the
bind mount options. Some of these options may not be recognised by the
cephfs mount. Hence added a new parameterin Storage Class for
- cephfs kernel mount options,
- ceph-fuse mount options
Ceph kernel mount options are different from ceph-fuse options, hence
added two different parameters.
Signed-off-by: Poornima G <pgurusid@redhat.com>
Ceph kernel client is more performant than ceph fuse client.
The kernel client has Quota support only in the kernel version >=4.17.
Hence use ceph kernel client when the kernel version is >=4.17.
Signed-off-by: Poornima G <pgurusid@redhat.com>
if both controller and nodeserver flags are set/unset
cephcsi will start both server,
if only one flag is set, it will start relavent
service.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
This will help user to check whats
the actual error. if the config file
is having issue or the clusterid is
not valid.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
It's CO responsibility to create the
stagingPath as per the CSI spec.
The CO SHALL ensure
// that the path is directory and that the process serving the
// request has `read` and `write` permission to that directory. The
// CO SHALL be responsible for creating the directory if it does not
// exist.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
... and not that of the FS subvolume group `csi`.
There is no reason for setting the mode of FS subvolume group `csi`
(a CephFS subdirectory) as 777. It's default mode is 755. It's
sufficient to set the mode of FS subvolumes within the subvolume group
to `777`.
Signed-off-by: Ramana Raja <rraja@redhat.com>
... instead of that of the `csi` subvolume group. The pool layout
specified via storage class's `pool` setting is a subvolume property
and not a subvolume group property. The `csi` subvolume group
may have subvolumes of different storage classes with different
pool layouts.
Fixes: #499
Signed-off-by: Ramana Raja <rraja@redhat.com>
Every Ceph CLI that is invoked at present passes the key via the
--key option, and hence is exposed to key being displayed on
the host using a ps command or such means.
This commit addresses this issue by stashing the key in a tmp
file, which is again created on a tmpfs (or empty dir backed by
memory). Further using such tmp files as arguments to the --keyfile
option for every CLI that is invoked.
This prevents the key from being visible as part of the argument list
of the invoked program on the system.
Fixes: #318
Signed-off-by: ShyamsundarR <srangana@redhat.com>
Currently, provisioner creates user for every volume and nodeplugin
uses this user to mount that volume. But nodeplugin and provisioner
already have admin credentials, hence using the admin credentials
to mount the volume and getting rid of user creation for each volume.
Signed-off-by: Poornima G <pgurusid@redhat.com>
in NodeStage RPC call we have to map the
device to the node plugin and make sure the
the device will be mounted to the global path
in nodeUnstage request unmount the device from
global path and unmap the device
if the volume mode is block we will be creating
a file inside a stageTargetPath and it will be
considered as the global path
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
update driver version and add git commit
to the image. This will help us to identify
what latest git commit image contains.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
Currently CephFs provisioner mounts the ceph filesystem
and creates a subdirectory as a part of provisioning the
volume. Ceph now supports commands to provision fs subvolumes,
hance modify the provisioner to use ceph mgr commands to
(de)provision fs subvolumes.
Signed-off-by: Poornima G <pgurusid@redhat.com>
As detailed in issue #279, current lock scheme has hash
buckets that are count of CPUs. This causes a lot of contention
when parallel requests are made to the CSI plugin. To reduce
lock contention, this commit introduces granular locks per
identifier.
The commit also changes the timeout for gRPC requests to Create
and Delete volumes, as the current timeout is 10s (kubernetes
documentation says 15s but code defaults are 10s). A virtual
setup takes about 12-15s to complete a request at times, that leads
to unwanted retries of the same request, hence the increased
timeout to enable operation completion with minimal retries.
Tests to create PVCs before and after these changes look like so,
Before:
Default master code + sidecar provisioner --timeout option set
to 30 seconds
20 PVCs
Creation: 3 runs, 396/391/400 seconds
Deletion: 3 runs, 218/271/118 seconds
- Once was stalled for more than 8 minutes and cancelled the run
After:
Current commit + sidecar provisioner --timeout option set to 30 sec
20 PVCs
Creation: 3 runs, 42/59/65 seconds
Deletion: 3 runs, 32/32/31 seconds
Fixes: #279
Signed-off-by: ShyamsundarR <srangana@redhat.com>
RBD plugin needs only a single ID to manage images and operations against a
pool, mentioned in the storage class. The current scheme of 2 IDs is hence not
needed and removed in this commit.
Further, unlike CephFS plugin, the RBD plugin splits the user id and the key
into the storage class and the secret respectively. Also the parameter name
for the key in the secret is noted in the storageclass making it a variant and
hampers usability/comprehension. This is also fixed by moving the id and the key
to the secret and not retaining the same in the storage class, like CephFS.
Fixes#270
Testing done:
- Basic PVC creation and mounting
Signed-off-by: ShyamsundarR <srangana@redhat.com>
* Enable all static-checks in golangci-lint
* Update golangci-lint version
* Fix issue found in golangci-lint
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
This is a part of the stateless set of commits for CephCSI.
This commit removes the dependency on config maps to store cephFS provisioned
volumes, and instead relies on RADOS based objects and keys, and required
CSI VolumeID encoding to detect the provisioned volumes.
Changes:
- Provide backward compatibility to provisioned volumes by older plugin versions (1.0.0 or older)
- Remove Create/Delete support for statically provisioned volumes (fixes#382)
- Added namespace support to RADOS OMaps and used the same to store RADOS CSI objects and keys in the CephFS metadata pool
- Added support to mention fsname for CephFS provisioning (fixes#359)
- Changed field name in CSI Identifier to 'location', to denote a pool or fscid
- Updated mounter cache to use new scheme
- Required Helm manifests are updated
- Required documentation and other manifests are updated
- Made driver option 'metadatastorage' as optional, as fresh installs do not need to specify the same
Testing done:
- Create/Mount/Delete PVC
- Create/Delete 5 PVCs
- Mount version 1.0.0 PVC
- Delete version 1.0.0 PV
- Mount Statically defined PV/PVC/Pod
- Mount Statically defined version 1.0.0 PV/PVC/Pod
- Delete Statically defined version 1.0.0 PV/PVC/Pod
- Node restart when mounted to test mountcache
- Use InstanceID other than 'default'
- RBD basic round of tests, as namespace is added to OMaps
- csitest against ceph-fs plugin
- NOTE: CephFS plugin still does not detect and address already created
volumes but of a different size
- Test not providing any value to the metadata storage parameter
Signed-off-by: ShyamsundarR <srangana@redhat.com>
