1
0
mirror of https://github.com/ceph/ceph-csi.git synced 2024-12-30 17:00:23 +00:00
Commit Graph

243 Commits

Author SHA1 Message Date
ShyamsundarR
2064e674a4 Addressed using k8s client APIs to fetch secrets
Based on the review comments addressed the following,
- Moved away from having to update the pod with volumes
when a new Ceph cluster is added for provisioning via the
CSI driver

- The above now used k8s APIs to fetch secrets
  - TBD: Need to add a watch mechanisim such that these
secrets can be cached and updated when changed

- Folded the Cephc configuration and ID/key config map
and secrets into a single secret

- Provided the ability to read the same config via mapped
or created files within the pod

Tests:
- Ran PV creation/deletion/attach/use using new scheme
StorageClass
- Ran PV creation/deletion/attach/use using older scheme
to ensure nothing is broken
- Did not execute snapshot related tests

Signed-off-by: ShyamsundarR <srangana@redhat.com>
2019-03-26 16:19:24 +00:00
ShyamsundarR
97f8c4b677 Provide options to pass in Ceph cluster-id
This commit provides the option to pass in Ceph cluster-id instead
of a MON list from the storage class.

This helps in moving towards a stateless CSI implementation.

Tested the following,
- PV provisioning and staging using cluster-id in storage class
- PV provisioning and staging using MON list in storage class

Did not test,
- snapshot operations in either forms of the storage class

Signed-off-by: ShyamsundarR <srangana@redhat.com>
2019-03-26 16:19:24 +00:00
王怀宗
b318964af5 issue
issue 

Goal

we try to solve when csi exit unexpect, the pod use cephfs pv can not auto recovery because lost mount relation until pod be killed and reschedule to other node. i think this is may be a problem. may be csi plugin can do more thing to remount the old path so when pod may be auto recovery when pod exit and restart, the old mount path can use.

NoGoal

Pod should exit and restart when csi plugin pod exit and mount point lost. if pod not exit will get error of **transport endpoint is not connected**.

implment logic

csi-plugin start:

	1. load all MountCachEntry  from node local dir
	2. check if volID exist in cluster, if no we ignore this entry, if yes continue
	3. check if stagingPath exist, if yes we mount the path
	4. check if all targetPath exist, if yes we binmount to staging path

NodeServer:

1. NodeStageVolume: add MountCachEntry on local dir include readonly attr and ceph secret
2. NodeStagePublishVolume: add pod bind mount path to MountCachEntry  and persist local dir
3. NodeStageunPublishVolume: remove pod bind mount path From MountCachEntry  and persist local dir
4. NodeStageunStageVolume: remove MountCachEntry  from local dir
2019-03-25 22:47:39 +08:00
John Griffith
d79ca5bc90 Merge branch 'csi-v1.0' into default_multiwrite_blockmode 2019-03-19 08:02:14 -06:00
j-griffith
6ec1196f47 Rework multi-node-multi-writer feature
This commit reverts the initial implementation of the
multi-node-multi-writer feature:
  commit: b5b8e46460

It replaces that implementation with a more restrictive version that
only allows multi-node-multi-writer for volumes of type `block`

With this change there are no volume parameters required in the stoarge
class, we also fail any attempt to create a file based device with
multi-node-multi-write being specified, this way a user doesn't have to
wait until they try and do the publish before realizing it doesn't work.
2019-03-18 10:07:06 -06:00
Róbert Vašek
a4dd845735
Merge pull request from Madhu-1/fix-222-1.0
update driver name as per csi spec
2019-03-14 06:38:13 +01:00
j-griffith
a164169fd3 Revert "Add multiNodeWritable option for RBD Volumes"
This reverts commit b5b8e46460.
2019-03-13 18:26:46 -06:00
Madhu Rajanna
d61a87b42e Fix driver name as per CSI spec
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-03-13 12:04:30 +05:30
Róbert Vašek
2a25666109
Merge pull request from Madhu-1/fix-return
Fix return error codes
2019-03-12 17:25:05 +01:00
Madhu Rajanna
dd41295900 store volume size in bytes in configmap
during volume creation we check volume size in
bytes, and even during listing of volumes and
snapshots we need to check size in bytes

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-03-12 18:37:10 +05:30
Madhu Rajanna
9f5ac5eeaa Fix return error codes
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-03-07 18:29:07 +05:30
Róbert Vašek
b07211739c
Merge pull request from Madhu-1/fix-image-size
Fix image size
2019-03-05 12:12:04 +01:00
Madhu Rajanna
16279eda78 Roundup volume size to Mib for rbd
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-03-04 19:17:28 +05:30
Madhu Rajanna
0fd091fa7f skip errcheck
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-03-04 19:02:10 +05:30
Madhu Rajanna
8f07c9efcc remove unused param from function
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-03-04 19:01:25 +05:30
j-griffith
72edf06916 Fix obsolete comment
Had a stray comment that got left behind during refactoring, move it
back to where it's relevant and reword it a bit to fit the finished
product.
2019-03-01 22:36:46 +00:00
j-griffith
b5b8e46460 Add multiNodeWritable option for RBD Volumes
This change adds the ability to define a `multiNodeWritable` option in
the Storage Class.

This change does a number of things:
1. Allow multi-node-multi-writer access modes if the SC options is
enabled
2. Bypass the watcher checks for MultiNodeMultiWriter Volumes
3. Maintains existing watcher checks for SingleNodeWriter access modes
regardless of the StorageClass option.

fix lint-errors
2019-03-01 21:59:57 +00:00
Huamin Chen
d938944528
Merge pull request from gman0/fuse-unmount-wait
cephfs: wait for FUSE to exit after unmount
2019-02-27 15:00:28 -05:00
gman
b779009770 parse ceph-fuse PID with regexp 2019-02-27 20:29:20 +01:00
Madhu Rajanna
6f4f148d3b remove glog
Signed-off-by: Madhu Rajanna <mrajanna@redhat.com>
2019-02-27 14:17:19 +05:30
Madhu Rajanna
fdc0d8255a move csi-common to ceph-csi
kubernetes/driver/csi-common is no
longer maintained.

Signed-off-by: Madhu Rajanna <mrajanna@redhat.com>
2019-02-27 14:17:19 +05:30
gman
3a0d048186 cephfs/volumemounter: unmountVolume now waits till the ceph-fuse daemon exits 2019-02-26 17:57:24 +01:00
Huamin Chen
dfcd1c33c3
Merge pull request from gman0/cephfs-concurrency-fix
cephfs concurrency fix
2019-02-26 09:40:32 -05:00
Madhu Rajanna
1cedbf0eee sort volume ID's for rbd volume list
looping over a map is not guaranteet
to be ordered.
we need to sort the volume ID's for
ListVolume rpc  for rbd plugin.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-02-26 14:35:16 +00:00
Madhu Rajanna
e861f12a60 Fix review comments
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-02-26 14:35:16 +00:00
Madhu Rajanna
3b320ef19e Add support of RBD list volumes
currently all the created volumes are
stored in the metadata store, so we
can use this information to support
list volumes.

Signed-off-by: Madhu Rajanna <mrajanna@redhat.com>
2019-02-26 14:35:16 +00:00
gman
2f8931315a don't attempt to delete mountpoint if unmount failed 2019-02-26 14:46:21 +01:00
gman
143003bcfd cephfs: added locks for {Create,Delete}Volume, NodeStageVolume 2019-02-26 11:06:25 +01:00
gman
60588d8968 cephfs/volume: create/delete-volume idempotency checks 2019-02-26 11:06:16 +01:00
gman
d12fdfd400 rbd: fixed metadata idempotency in DeleteVolume; DeleteSnapshot should assume the snapshot to be already deleted if metadata doesn't exist 2019-02-25 18:09:21 +01:00
gman
ce3affcc6a cephfs: DeleteVolume should assume the volume to be already deleted if metadata doesn't exist 2019-02-25 18:07:28 +01:00
gman
e5dbea15d3 util/cachepersister: check and return CacheEntryNotFound error in Get() 2019-02-25 18:05:20 +01:00
Huamin Chen
e627a6166c
Merge pull request from Madhu-1/skip_cred_log
remove  secret and key  from logging
2019-02-25 08:04:38 -05:00
Madhu Rajanna
68e69502e8 remove secret and key from logging
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-02-21 17:05:24 +05:30
gman
0235b9c249 k8s metadata cache: delete shouldn't fail on NotFound errors 2019-02-20 20:20:44 +01:00
Róbert Vašek
bc256aa755
Merge pull request from Madhu-1/clean-up
cleanup: remove duplicate code
2019-02-19 15:50:46 +01:00
Madhu Rajanna
fd4c019aba cleanup: remove duplicate code
Signed-off-by: Madhu Rajanna <mrajanna@redhat.com>
2019-02-19 13:44:10 +05:30
Madhu Rajanna
e91a59d2f9 RBD: Remove dead code
during volume creation we are validating
that volume name cannot be empty,removing
this check as we are not going to hit
this case

Fixes: 

Signed-off-by: Madhu Rajanna <mrajanna@redhat.com>
2019-02-19 12:17:40 +05:30
Madhu Rajanna
6fbde110cd skip redundant error check
correct misspelled  word

Signed-off-by: Madhu Rajanna <mrajanna@redhat.com>
2019-02-18 17:16:59 +05:30
gman
49f5d4a54e fix lint error 2019-02-14 21:31:52 +00:00
gman
1803a1be97 rbd: don't delete volume/snapshot if metadata creation fails 2019-02-14 21:31:52 +00:00
gman
8223ae325b addressed review comments 2019-02-14 13:55:51 +00:00
gman
96bf4a98bd cephfs: don't need to store keyrings anymore 2019-02-14 13:55:51 +00:00
gman
8e371f62fa cephfs: ceph-fuse mimic supports passing keys via args, let's use it 2019-02-14 13:55:51 +00:00
gman
b3944f3121 cephfs exec: read stdout and stderr separately 2019-02-14 13:55:51 +00:00
gman
892d65d387 added StripSecretInArgs in pkg/util 2019-02-14 13:55:51 +00:00
gman
a63b06a620 cephfs: specify monitors explicitly 2019-02-13 14:09:33 +00:00
gman
9cd0786569 cephfs/volume: fixed incorrect path for unmountAndRemove 2019-02-12 17:39:54 +01:00
gman
6099f142f0 moved klog initialization into pkg/util package 2019-02-12 16:31:55 +01:00
Huamin Chen
a195856104
Merge pull request from Madhu-1/remove-glog
Remove glog from Gopkg.toml
2019-02-08 07:23:13 -05:00
Huamin Chen
9c4e5543c9
Merge pull request from Madhu-1/log-req
Avoid logging secrets in request
2019-02-08 07:21:19 -05:00
Madhu Rajanna
bf7ec10fea Remove glog from Gopkg.toml
replace klog with glog

Signed-off-by: Madhu Rajanna <mrajanna@redhat.com>
2019-02-08 13:51:22 +05:30
Madhu Rajanna
a5f2635bc0 avoid logging secrets in request
Signed-off-by: Madhu Rajanna <mrajanna@redhat.com>
2019-02-08 13:20:21 +05:30
Madhu Rajanna
f670c62526 Remove userID and key logging from rbd
Fixes: 
2019-02-08 12:40:48 +05:30
Humble Chirammal
c0712db08a migrate util package to klog from glog.
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2019-02-05 12:09:04 +00:00
Humble Chirammal
cb77ff5d87 Migrate rbd package to klog from glog
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2019-02-05 12:09:04 +00:00
Humble Chirammal
c9da8469ad migrate cephfs code to use klog instead of glog
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2019-02-05 12:09:04 +00:00
Madhu Rajanna
5983fedddb Fix issues found in gometalinter static check
Signed-off-by: Madhu Rajanna <mrajanna@redhat.com>
2019-02-04 10:44:37 +05:30
Madhu Rajanna
74796bd57b Fix review comment
Signed-off-by: Madhu Rajanna <mrajanna@redhat.com>
2019-01-29 18:29:07 +05:30
Madhu Rajanna
03d93219d7 Fix metalinter issue
Signed-off-by: Madhu Rajanna <mrajanna@redhat.com>
2019-01-29 11:37:03 +05:30
Madhu Rajanna
50ba8ed446 Fix gometalinter issues
Signed-off-by: Madhu Rajanna <mrajanna@redhat.com>
2019-01-29 11:24:36 +05:30
Madhu Rajanna
ca2e475296 Fix gometalinter issues
Signed-off-by: Madhu Rajanna <mrajanna@redhat.com>
2019-01-29 11:23:50 +05:30
Madhu Rajanna
7a0c233c27 Fix issues found in gometalinter
Signed-off-by: Madhu Rajanna <mrajanna@redhat.com>
2019-01-29 11:20:35 +05:30
Madhu Rajanna
008c82c1e7 Fix gometalinter issues
Signed-off-by: Madhu Rajanna <mrajanna@redhat.com>
2019-01-29 11:20:35 +05:30
Madhu Rajanna
25642fe404 Add method comments
Signed-off-by: Madhu Rajanna <mrajanna@redhat.com>
2019-01-29 11:20:35 +05:30
Madhu Rajanna
b71a9bfe40 Fix issue with error handling in metadata store
Signed-off-by: Madhu Rajanna <mrajanna@redhat.com>
2019-01-28 10:26:57 +05:30
Huamin Chen
aa4fce2c8b
Merge pull request from Madhu-1/fix-golint
Fix golint
2019-01-27 10:04:42 -05:00
Madhu Rajanna
48ca11336e Fix snapshot time during snapshot create
Fixes: 

Signed-off-by: Madhu Rajanna <mrajanna@redhat.com>
2019-01-25 19:09:35 +05:30
Madhu Rajanna
7fcbdfb945 Fix template to add userID
Fix Node server

Signed-off-by: Madhu Rajanna <mrajanna@redhat.com>
2019-01-25 14:20:32 +05:30
Madhu Rajanna
1a246f7bf3 simplify the error check
Signed-off-by: Madhu Rajanna <mrajanna@redhat.com>
2019-01-25 14:16:03 +05:30
Madhu Rajanna
36f99e36ca Fix unparam issues
Signed-off-by: Madhu Rajanna <mrajanna@redhat.com>
2019-01-25 14:16:03 +05:30
Madhu Rajanna
284c5801c3 Fix golint issue
pkg/rbd/rbd.go:67:65⚠️ exported func NewNodeServer
returns unexported type *rbd.nodeServer, which can be
annoying to use (golint)

Signed-off-by: Madhu Rajanna <mrajanna@redhat.com>
2019-01-25 14:16:03 +05:30
Madhu Rajanna
4c564207c6 Fix error checking issue in json Decode
Signed-off-by: Madhu Rajanna <mrajanna@redhat.com>
2019-01-25 14:15:25 +05:30
Madhu Rajanna
1d11d0acc3 Fix lint issues
Signed-off-by: Madhu Rajanna <mrajanna@redhat.com>
2019-01-25 14:15:25 +05:30
Madhu Rajanna
5eb1974e38 Fix vetshadow issues
Signed-off-by: Madhu Rajanna <mrajanna@redhat.com>
2019-01-25 14:15:25 +05:30
Madhu Rajanna
75beccd8c0 Remove unused variable
Signed-off-by: Madhu Rajanna <mrajanna@redhat.com>
2019-01-25 14:14:48 +05:30
Madhu Rajanna
49c8fe5d50 Remove conversion which is not required
Signed-off-by: Madhu Rajanna <mrajanna@redhat.com>
2019-01-25 14:14:48 +05:30
Madhu Rajanna
1c03334a90 Add constant for rbd and ndb
Signed-off-by: Madhu Rajanna <mrajanna@redhat.com>
2019-01-25 14:14:48 +05:30
Madhu Rajanna
fa78c00535 update struct names and struct fiels
it wont be meaningful to call cephfs.NewcephfsDriver()
to get a new driver, it will be better if we call
cephfs.GetNewDriver() which returns the cephfs driver
object.

same goes for rbd also

Signed-off-by: Madhu Rajanna <mrajanna@redhat.com>
2019-01-25 14:14:48 +05:30
Madhu Rajanna
15b5b0112e rename Id to ID to fix lint issue
Signed-off-by: Madhu Rajanna <mrajanna@redhat.com>
2019-01-25 14:14:48 +05:30
Madhu Rajanna
8f2941dd47 Remove code complexity
Signed-off-by: Madhu Rajanna <mrajanna@redhat.com>
2019-01-25 14:13:04 +05:30
Madhu Rajanna
de1f099680 Remove unused struct fields
Signed-off-by: Madhu Rajanna <mrajanna@redhat.com>
2019-01-25 14:10:52 +05:30
Madhu Rajanna
9f76f6bd59 Remove dead code
Signed-off-by: Madhu Rajanna <mrajanna@redhat.com>
2019-01-25 14:10:52 +05:30
Huamin Chen
c6c496ff59 switch to node registrar 2019-01-22 14:46:41 -05:00
Huamin Chen
0151792684 review feedback: make monValueFromSecret override monitors if both are set
Signed-off-by: Huamin Chen <hchen@redhat.com>
2019-01-21 09:21:03 -05:00
Huamin Chen
db463edeef allow ceph mon stored in secret so when mon changes, cephfs driver can get latest mons and override old ones
Signed-off-by: Huamin Chen <hchen@redhat.com>
2019-01-18 10:27:48 -05:00
Huamin Chen
48407e2484 add csi volume device mount path to csi plugin
Signed-off-by: Huamin Chen <hchen@redhat.com>
2019-01-17 08:57:18 -05:00
Huamin Chen
263c45bb45 enable csi block; use canary external-provisioner image to pick up block volume provisioning
Signed-off-by: Huamin Chen <hchen@redhat.com>
2019-01-16 13:52:45 -05:00
Masaki Kimura
0e60dabca3 Move resolving bind mount logic from k8s 2019-01-16 13:00:48 -05:00
Masaki Kimura
ea75a9d162 Fix pv deletion issue caused by not dettaching due to wrong mount count 2019-01-16 12:51:13 -05:00
Masaki Kimura
5867d495fd Change csi rbd to create/delete targetPath for publish/unpublish 2019-01-16 12:50:17 -05:00
Masaki Kimura
165b82a44c Add block supports to rbd driver 2019-01-16 12:49:02 -05:00
Madhu Rajanna
e42e66ff30 user errors.New if error formatting is not required
Signed-off-by: Madhu Rajanna <mrajanna@redhat.com>
2019-01-16 18:54:13 +05:30
Madhu Rajanna
17f5c0a7ce use Errorf is error formatting is required
Signed-off-by: Madhu Rajanna <mrajanna@redhat.com>
2019-01-16 18:54:06 +05:30
Madhu Rajanna
20af5afcab Fix golint issues
Signed-off-by: Madhu Rajanna <mrajanna@redhat.com>
2019-01-16 18:33:38 +05:30
Madhu Rajanna
9ddc265c10 reject block volume creation in cephfs
Signed-off-by: Madhu Rajanna <mrajanna@redhat.com>
2019-01-16 18:17:14 +05:30
Huamin Chen
aed7506d88 fix merge leftovers; use canary driver-registrar image, as v1.0.0 is not hosted in quay.io
Signed-off-by: Huamin Chen <hchen@redhat.com>
2019-01-15 13:31:06 -05:00
Huamin Chen
e46099a504 reconcile merge
Signed-off-by: Huamin Chen <hchen@redhat.com>
2019-01-15 16:20:41 +00:00
Huamin Chen
85b8415024 Merge branch 'master' into master-to-1.0 2019-01-15 16:15:30 +00:00
mickymiek
b387daaabf remove useless comment 2019-01-14 20:15:09 +00:00