Commit Graph

995 Commits

Author SHA1 Message Date
Vasyl Purchel
419ad0dd8e Adds per volume encryption with Vault integration
- adds proposal document for PVC encryption from PR448
- adds per-volume encription by generating encryption passphrase
  for each volume and storing it in a KMS
- adds HashiCorp Vault integration as a KMS for encryption passphrases
- avoids encrypting volume second time if it was already encrypted but
  no file system created
- avoids unnecessary checks if volume is a mapped device when encryption
  was not requested
- prevents resizing encrypted volumes (it is not currently supported)
- prevents creating snapshots from encrypted volumes to prevent attack
  on encryption key (security guard until re-encryption of volumes
  implemented)

Signed-off-by: Vasyl Purchel vasyl.purchel@workday.com
Signed-off-by: Andrea Baglioni andrea.baglioni@workday.com

Fixes #420
Fixes #744
2020-02-05 05:18:56 +00:00
Jakub Kuzelka
1adef00c86 Repair typo in host-mount and host-dev definition. It was swithced between each other and actually there is not possible to deploy nodeplugin-daemonset via helm chart. 2020-01-29 16:00:10 +00:00
ShyamsundarR
35e8c3b3a5 CephFS: Added ENOENT checks for possible missing volumes
Added checks in DeleteVolume RPC, for image missing errors, and
taking appropriate actions to cleanup the CSI reservations.

Further removed forcing a volume purge, and instead added checks
for missing volume errors in purgeVolume.

This should now fix issues where an continuation of an interrupted
DeleteVolume call, that only deleted the backing volume, will
proceed and not error out.

Signed-off-by: ShyamsundarR <srangana@redhat.com>
2020-01-29 10:05:13 +00:00
ShyamsundarR
b93ed21fe8 Add e2e test case for DeleteVolume success on backend volume deletion
If a backend volume is deleted, DeleteVolume call for the same should
succeed, detecting the image is missing and delete the related OMaps.

This commit adds a test case to ensure this is occuring correctly.

Updates #474

Signed-off-by: ShyamsundarR <srangana@redhat.com>
2020-01-29 10:05:13 +00:00
Madhu Rajanna
eb2fb9233b Add run hostpath to daemonset pods
`/run/mount` need to be share between host and
csi-plugin containers for `/run/mount/utab`

this is required to ensures that the network
is not stopped prior to unmounting the network devices.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-01-28 16:50:18 +00:00
Madhu Rajanna
881f59d142 Add _netdev as default mount options in plugin
This values will be added at both nodestage
and nodepublish for rbd, nbd and ceph kernel client.

As cephfs fuse doesnot support this value,
this is added only during the nodepublish.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-01-28 16:50:18 +00:00
Oguz Kilcan
aadce54b2f Added PodSecurityPolicy support 2020-01-22 08:19:42 +00:00
Niels de Vos
7ec8e68b67 examples/rbd-block: use a container image that is supported
Fedora 26 has been End-Of-Life for a long time already, is should not be
used anymore. Instead use the latest CentOS image that gets regular
updates. The Ceph base image used by Ceph-CSI is also based on CentOS,
so uses would be familiar with the available tools.

Also use "sleep infinity" instead of a weird 'tail -f /dev/null' hack.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-01-21 13:30:45 +00:00
Madhu Rajanna
520ceb6dcb Fix allignment issue in shellscript
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-01-20 17:33:03 +05:30
Madhu Rajanna
50a84fb930 Update mergify to release-v2.0
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-01-20 17:33:03 +05:30
Niels de Vos
a0ef9da390 rbd: only load nbd module if not available yet
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-01-20 11:22:13 +00:00
wilmardo
f04af5742d refact: Remove Kubernetes 1.13.x support
Signed-off-by: wilmardo <info@wilmardenouden.nl>
2020-01-20 10:32:30 +00:00
Madhu Rajanna
70c709155c Add 13.0.0 Mimic supported version to readme
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-01-17 13:45:20 +00:00
Madhu Rajanna
5596cce8d8 update Metrics supported version in Readme
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-01-17 13:45:20 +00:00
Madhu Rajanna
8ec5aae932 Update readme for resize metrics
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-01-17 13:45:20 +00:00
Madhu Rajanna
369fdb5051 Doc for cephfs expand PVC
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-01-17 13:45:20 +00:00
Humble Chirammal
7df51846da This commit adds "k8s.io/kubernetes/test/e2e/framework/config"
and its functions in E2E.

update vendor packages
log dismounter command output
use kube v1.17.1 in dependency

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-01-17 12:06:02 +00:00
Humble Chirammal
2d0ed298e3 Update framework imports and its functions.
Signed-off-by: Cedric Hauber <hauber.c@gmail.com>
2020-01-17 12:06:02 +00:00
Humble Chirammal
7c96c5c567 Remove unused import from e2e utils
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-01-17 12:06:02 +00:00
Humble Chirammal
5e6aebee58 Exclude snapshot changes
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-01-17 12:06:02 +00:00
Humble Chirammal
e8e9208987 Add k8s.io/utils/exec to Gopkg.*
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-01-17 12:06:02 +00:00
Humble Chirammal
3af1e26d7c Update to kube v1.17
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-01-17 12:06:02 +00:00
sophal
327fcd1b1b Rook deploy script, Adding rbd_pool check and cephfilesystem check. 2020-01-14 08:51:27 +00:00
Madhu Rajanna
6a8ddad669 Add Ceph-csi Upgrade documentation
Added Upgrade doc from v1.2.x to v2.0.0

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-01-14 07:50:52 +00:00
Humble Chirammal
866bc7da21 Add resize doc for both filesystem and block mode PVCs
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-01-13 14:53:08 +00:00
Madhu Rajanna
e0cc7740f6 CSI: run all containers as privileged in daemonset pods
On systems with SELinux enabled, non-privileged containers
can't access data of privileged containers. Since the socket
is exposed by privileged containers, all sidecars must be
privileged too. This is needed only for containers running
in daemonset as we are using bidirectional mounts in daemonset

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-01-13 13:21:29 +00:00
Madhu Rajanna
fbda8cc4ca Use EmptyDir to store provisioner socket
currently, we are making use of host path directory
to store the provisioner socket, as this
the socket is not needed by anyone else other than
containers inside the provisioner pod using the
empty directory to store this socket is the best option.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-01-13 13:21:29 +00:00
Madhu Rajanna
a4f6a3407f Add ContentSource to the CreateVolume response
if PVC is created from a snapshot, external-provisioner
expects the volume ContentSource to be set in Create
VolumeResponse

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-01-10 09:20:45 +00:00
Yibo Cai
4b8b52e0de Support Arm64 image
Update CI merge job to build and push Arm64 image to
quay.io/cephcsi/cephcsi:version-arm64.

Add CI PR job running on Travis Arm64 nodes to make sure cephcsi
compiles successfully on Arm64.

No CI test job is availabe for Arm64 now due to below issues
- k8s-csi sidecar images for Arm64 are not available
- Travis Arm64 CI job runs inside unprivileged LXD which blocks
  launching minikube test environment

Signed-off-by: Yibo Cai <yibo.cai@arm.com>
2020-01-09 09:53:50 +00:00
Wong Hoi Sing Edison
543360ee00 Upgrade: csi-attacher from v1.2.0 to v2.1.0
See https://github.com/kubernetes-csi/external-attacher/releases/tag/v2.1.0
See https://github.com/kubernetes-csi/external-attacher/blob/v2.1.0/CHANGELOG-2.1.md
2020-01-07 14:27:29 +00:00
Chin-Ya Huang
47f89889ef Fix typo for comment in charts
* ceph-csi-cephfs
* ceph-csi-rbd

Signed-off-by: Chin-Ya Huang <chin-ya.huang@suse.com>
2020-01-07 07:52:32 +00:00
Wong Hoi Sing Edison
202a2a7200 Upgrade: csi-snapshotter from v1.2.1 to v1.2.2
See https://github.com/kubernetes-csi/external-snapshotter/releases/tag/v1.2.2
See https://github.com/kubernetes-csi/external-snapshotter/blob/v1.2.2/CHANGELOG-1.2.md
2020-01-06 12:36:44 +00:00
Wong Hoi Sing Edison
f37bdfdd44 Upgrade: csi-node-driver-registrar from v1.1.0 to v1.2.0
See https://github.com/kubernetes-csi/node-driver-registrar/releases/tag/v1.2.0
See https://github.com/kubernetes-csi/node-driver-registrar/blob/v1.2.0/CHANGELOG-1.2.md
2020-01-06 07:48:41 +00:00
Madhu Rajanna
4d28a981fc Remove hard-coded UpdateStrategy from templates
Provided an option to specify the UpdateStrategy
in helm  charts.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-01-05 08:05:06 +00:00
Wong Hoi Sing Edison
74cb18bd28 Upgrade: csi-resizer from v0.3.0 to v0.4.0
See https://github.com/kubernetes-csi/external-resizer/releases/tag/v0.4.0
See https://github.com/kubernetes-csi/external-resizer/blob/v0.4.0/CHANGELOG-0.4.md
2020-01-05 07:21:12 +00:00
Wong Hoi Sing Edison
3e656769b7 Update csi-provisioner from v1.3.0 to v1.4.0
See https://github.com/kubernetes-csi/external-provisioner/releases/tag/v1.4.0
See https://github.com/kubernetes-csi/external-provisioner/blob/v1.4.0/CHANGELOG-1.4.md
2020-01-02 15:53:07 +00:00
Madhu Rajanna
f69ec085cc Improve logging while waiting for deployment to complete
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-12-31 06:10:40 +00:00
Madhu Rajanna
ad7e5ca301 Add E2E for block PVC resize
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-12-31 06:10:40 +00:00
Madhu Rajanna
374e47046f Move resize E2E to new resize.go file
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-12-31 06:10:40 +00:00
Madhu Rajanna
3394432371 Fix block resize issue in RBD
For the filesystem resize the stagingPath will
be passed to NodeExpandVolume and for the block
resize the targetpath will be passed.
Added code to handle resize for both block and
file system resize

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-12-31 06:10:40 +00:00
Sébastien Bernard
40b04d2f3a Add missing env for namespace. 2019-12-20 13:59:15 +00:00
Madhu Rajanna
7d15992769 Enable logging in E2E if test fails
source: https://github.com/kubernetes/kubernetes/blob/master/test/e2e/framework/kubectl/kubectl_utils.go
kubectlLogPod function is not exposed in above code so copied it.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-12-18 12:35:17 +00:00
Madhu Rajanna
b4f20e941c Enable Block E2E for rbd
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-12-18 12:35:17 +00:00
Madhu Rajanna
72e3767594 Replace Warningf and Infof with Errorf
incase of err, log as Errorf instead of
warningf and Infof

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-12-17 14:25:34 +00:00
Madhu Rajanna
85960b6571 Add ID based logging for ExpandVolume
Updated logging to log ReqID

Fixes: #732

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-12-17 14:25:34 +00:00
Madhu Rajanna
dcafdb519e discard umount error if directory is not mounted
if the directory is not mounted return nil
during umount of mountPoint

Discard error if error is os.IsNotExist

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-12-17 13:33:41 +00:00
Madhu Rajanna
b849b7daaa Fix leader election flag in deployment files
Fixes: https://github.com/ceph/ceph-csi/issues/748

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-12-17 12:19:01 +00:00
Madhu Rajanna
f281eba475 Validate rbd image name in NodeExpand
Added one more validation in NodeExpand for
the volume name from the request and the image
name stored in the staging path metadata.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-12-17 11:33:50 +00:00
Vasyl Purchel
166eaf700f Adds PVC encryption with LUKS
Adds encryption in StorageClass as a parameter. Encryption passphrase is
stored in kubernetes secrets per StorageClass. Implements rbd volume
encryption relying on dm-crypt and cryptsetup using LUKS extension

The change is related to proposal made earlier. This is a first part of
the full feature that adds encryption with passphrase stored in secrets.

Signed-off-by: Vasyl Purchel vasyl.purchel@workday.com
Signed-off-by: Andrea Baglioni andrea.baglioni@workday.com
Signed-off-by: Ioannis Papaioannou ioannis.papaioannou@workday.com
Signed-off-by: Paul Mc Auley paul.mcauley@workday.com
Signed-off-by: Sergio de Carvalho sergio.carvalho@workday.com
2019-12-16 08:12:44 +00:00
Humble Chirammal
7c8e66e427 Add resize check for XFS formatted FS
Lock out parellel requests against same volumeID
Remove pod after resize and validation in E2E

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2019-12-13 12:40:12 +00:00