Commit Graph

1920 Commits

Author SHA1 Message Date
Madhu Rajanna
a9174dd953 Fix logging if the rbd manager command is supported
if there is an error when adding the rbd task
we are logging the output which is empty. This
PR logs the error if the rbd task is supported
and there is an error.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-03-16 11:49:31 +00:00
Madhu Rajanna
b4e6504e9b Fix mountoption issue in rbd
use mountoptions when mounting rbd to stagingpath
in stagevolume request, add E2E for mount options

fixes: #846
updates: #757

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-03-13 14:33:56 +00:00
Niels de Vos
40d0d5d291 rbd: drop references to ImageFormat
librbd only supports ImageFormat 2. It is not expected that anyone has a
different version of the format in container environments.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-03-11 16:09:10 +00:00
Niels de Vos
8dc3600899 rbd: use go-ceph API for creating RBD images
This is the initial step for improving performance during provisioning
of CSI volumes backed by RBD.

While creating a volume, an existing connection to the Ceph cluster is
used from the ConnPool. This should speed up the creation of a batch of
volumes significantly.

Updates: #449
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-03-11 16:09:10 +00:00
Niels de Vos
90f81516ee util/conn_pool: add tests for ConnPool
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-03-11 16:09:10 +00:00
Niels de Vos
397825c665 util: add ConnPool for connection re-use
By using the ConnPool it is not needed to re-connect every time to the
Ceph cluster when (rbd) operations are executed through the go-ceph/rbd
API.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-03-11 16:09:10 +00:00
Niels de Vos
ba99275f90 dep: add github.com/ceph/go-ceph for rbd API
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-03-11 16:09:10 +00:00
Niels de Vos
3226b17d08 enable building with cgo for go-ceph usage
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-03-11 16:09:10 +00:00
Niels de Vos
66b73e3682 travis: install Ceph development packages for go-ceph
The Ceph community does not guarantee that arm64 packages get build and
published with each release. In order to prevent falling back to ancient
Ceph versions from ubuntu-ports, add an unsigned repository with Ceph
Nautilus 14.2.5.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-03-11 16:09:10 +00:00
Reinier Schoof
ca8dd2d8f2 use VolSize from rbdVolume instead of separate parameter 2020-03-10 11:34:53 +00:00
Reinier Schoof
8da7e4bbf9 removed unreachable code path 2020-03-10 11:34:53 +00:00
Reinier Schoof
3af5e0619f created struct for keeping the state of a staging transaction
this way extending transaction rollbacks is easier

Signed-off-by: Reinier Schoof <reinier@skoef.nl>
2020-03-10 11:34:53 +00:00
Jonas Rutishauser
9d7b50dccb Added forcecephkernelclient as startup parameter
Support #664 in the helm chart.

Signed-off-by: Jonas Rutishauser <jonas.rutishauser@alumni.ethz.ch>
2020-03-03 09:50:42 +05:30
Madhu Rajanna
128f3fc2cf check subvolume present in backend
If a CreateVolume call is interrupted,
post creating the required CSI journal entries,
but prior to creating the backing CephFS subvolume,
then a subsequent CreateVolume call will return
a valid response with a VolumeID that has
it's backing image missing. This PR adds a check
for backend image, if image notfound it deletes the
reserved keys in omap.

fixes #839

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-02-25 11:46:04 +00:00
Reinier Schoof
a4532fafd0 added volumeNamePrefix and snapshotNamePrefix as parameters for storageClass
this allows administrators to override the naming prefix for both volumes and snapshots
created by the rbd plugin.

Signed-off-by: Reinier Schoof <reinier@skoef.nl>
2020-02-25 05:03:51 +00:00
Madhu Rajanna
8163552b81 Add doc for rbd static pvc support
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-02-18 12:34:46 +00:00
Madhu Rajanna
1ec2bc47e3 Add E2E for rbd static PVC
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-02-18 12:34:46 +00:00
Madhu Rajanna
9f15dded6d Add support for static rbd volumes
currently its not possible to create
and mount static PVC to a application pod
using rbd csi driver. This PR adds the support
for static PVC in ceph-csi driver for rbd.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-02-18 12:34:46 +00:00
Madhu Rajanna
0f80ec0664 Return err from nodeserver
getVolumeNameByID function is already
returrning the status.Error return the err
as it is.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-02-18 12:34:46 +00:00
Madhu Rajanna
90235c32f8 Fix branch name in notes.txt
currently notes.txt container old branch name.
This PR updated the branch name with release

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-02-13 21:36:27 +00:00
Madhu Rajanna
d02dfe2dfe Remove unwanted RBAC rules from ceph-csi
There are currently unwanted RBAC permission
is given for ceph-csi, This PR reduces removes
such unwanted RBAC resources.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-02-13 21:36:27 +00:00
Madhu Rajanna
8dcb6a6105 Handle Delete operation if pool not found
If the backend rbd or cephfs pool is already deleted
we need to return success to the  DeleteVolume RPC
call to make it idempotent.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-02-11 15:58:15 +00:00
Madhu Rajanna
034b123478 Remove mount cache for cephfs
PR #282 introduces the mount cache to
solve cephfs fuse mount issue when cephfs plugin pod
restarts .This is not working as intended. This PR removes
the code for maintainability.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-02-11 15:11:21 +00:00
Vasyl Purchel
669dc4536f Reduce encryption KMS configuration SC parameters
* moves KMS type from StorageClass into KMS configuration itself
 * updates omapval used to identify KMS to only it's ID without the type

why?

1. when using multiple KMS configurations (not currently supported)
automated parsing of kms configuration will be failing because some
entries in configs won't comply with the requested type
2. less options are needed in the StorageClass and less data used to
identify the KMS

Signed-off-by: Vasyl Purchel vasyl.purchel@workday.com
Signed-off-by: Andrea Baglioni andrea.baglioni@workday.com
2020-02-10 15:21:11 +00:00
sophal
1695c6965d Add printing failed message when timeout
Signed-off-by: sophal <sophalHong@github.com>
2020-02-07 13:05:30 +00:00
Vasyl Purchel
419ad0dd8e Adds per volume encryption with Vault integration
- adds proposal document for PVC encryption from PR448
- adds per-volume encription by generating encryption passphrase
  for each volume and storing it in a KMS
- adds HashiCorp Vault integration as a KMS for encryption passphrases
- avoids encrypting volume second time if it was already encrypted but
  no file system created
- avoids unnecessary checks if volume is a mapped device when encryption
  was not requested
- prevents resizing encrypted volumes (it is not currently supported)
- prevents creating snapshots from encrypted volumes to prevent attack
  on encryption key (security guard until re-encryption of volumes
  implemented)

Signed-off-by: Vasyl Purchel vasyl.purchel@workday.com
Signed-off-by: Andrea Baglioni andrea.baglioni@workday.com

Fixes #420
Fixes #744
2020-02-05 05:18:56 +00:00
Jakub Kuzelka
1adef00c86 Repair typo in host-mount and host-dev definition. It was swithced between each other and actually there is not possible to deploy nodeplugin-daemonset via helm chart. 2020-01-29 16:00:10 +00:00
ShyamsundarR
35e8c3b3a5 CephFS: Added ENOENT checks for possible missing volumes
Added checks in DeleteVolume RPC, for image missing errors, and
taking appropriate actions to cleanup the CSI reservations.

Further removed forcing a volume purge, and instead added checks
for missing volume errors in purgeVolume.

This should now fix issues where an continuation of an interrupted
DeleteVolume call, that only deleted the backing volume, will
proceed and not error out.

Signed-off-by: ShyamsundarR <srangana@redhat.com>
2020-01-29 10:05:13 +00:00
ShyamsundarR
b93ed21fe8 Add e2e test case for DeleteVolume success on backend volume deletion
If a backend volume is deleted, DeleteVolume call for the same should
succeed, detecting the image is missing and delete the related OMaps.

This commit adds a test case to ensure this is occuring correctly.

Updates #474

Signed-off-by: ShyamsundarR <srangana@redhat.com>
2020-01-29 10:05:13 +00:00
Madhu Rajanna
eb2fb9233b Add run hostpath to daemonset pods
`/run/mount` need to be share between host and
csi-plugin containers for `/run/mount/utab`

this is required to ensures that the network
is not stopped prior to unmounting the network devices.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-01-28 16:50:18 +00:00
Madhu Rajanna
881f59d142 Add _netdev as default mount options in plugin
This values will be added at both nodestage
and nodepublish for rbd, nbd and ceph kernel client.

As cephfs fuse doesnot support this value,
this is added only during the nodepublish.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-01-28 16:50:18 +00:00
Oguz Kilcan
aadce54b2f Added PodSecurityPolicy support 2020-01-22 08:19:42 +00:00
Niels de Vos
7ec8e68b67 examples/rbd-block: use a container image that is supported
Fedora 26 has been End-Of-Life for a long time already, is should not be
used anymore. Instead use the latest CentOS image that gets regular
updates. The Ceph base image used by Ceph-CSI is also based on CentOS,
so uses would be familiar with the available tools.

Also use "sleep infinity" instead of a weird 'tail -f /dev/null' hack.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-01-21 13:30:45 +00:00
Madhu Rajanna
520ceb6dcb Fix allignment issue in shellscript
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-01-20 17:33:03 +05:30
Madhu Rajanna
50a84fb930 Update mergify to release-v2.0
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-01-20 17:33:03 +05:30
Niels de Vos
a0ef9da390 rbd: only load nbd module if not available yet
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-01-20 11:22:13 +00:00
wilmardo
f04af5742d refact: Remove Kubernetes 1.13.x support
Signed-off-by: wilmardo <info@wilmardenouden.nl>
2020-01-20 10:32:30 +00:00
Madhu Rajanna
70c709155c Add 13.0.0 Mimic supported version to readme
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-01-17 13:45:20 +00:00
Madhu Rajanna
5596cce8d8 update Metrics supported version in Readme
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-01-17 13:45:20 +00:00
Madhu Rajanna
8ec5aae932 Update readme for resize metrics
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-01-17 13:45:20 +00:00
Madhu Rajanna
369fdb5051 Doc for cephfs expand PVC
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-01-17 13:45:20 +00:00
Humble Chirammal
7df51846da This commit adds "k8s.io/kubernetes/test/e2e/framework/config"
and its functions in E2E.

update vendor packages
log dismounter command output
use kube v1.17.1 in dependency

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-01-17 12:06:02 +00:00
Humble Chirammal
2d0ed298e3 Update framework imports and its functions.
Signed-off-by: Cedric Hauber <hauber.c@gmail.com>
2020-01-17 12:06:02 +00:00
Humble Chirammal
7c96c5c567 Remove unused import from e2e utils
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-01-17 12:06:02 +00:00
Humble Chirammal
5e6aebee58 Exclude snapshot changes
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-01-17 12:06:02 +00:00
Humble Chirammal
e8e9208987 Add k8s.io/utils/exec to Gopkg.*
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-01-17 12:06:02 +00:00
Humble Chirammal
3af1e26d7c Update to kube v1.17
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-01-17 12:06:02 +00:00
sophal
327fcd1b1b Rook deploy script, Adding rbd_pool check and cephfilesystem check. 2020-01-14 08:51:27 +00:00
Madhu Rajanna
6a8ddad669 Add Ceph-csi Upgrade documentation
Added Upgrade doc from v1.2.x to v2.0.0

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-01-14 07:50:52 +00:00
Humble Chirammal
866bc7da21 Add resize doc for both filesystem and block mode PVCs
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-01-13 14:53:08 +00:00