Commit Graph

79 Commits

Author SHA1 Message Date
Madhu Rajanna
0da4bd5151 start controller or node server based on config
if both controller and nodeserver flags are set/unset
cephcsi will start both server,

if only one flag is set, it will start relavent
service.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-08-19 06:11:43 +00:00
Humble Chirammal
0fc7f4513b Snashotter update
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2019-08-19 05:06:42 +00:00
wilmardo
0a90762970 fix: Adds liveness sidecar to v1.14+ helm charts
Signed-off-by: wilmardo <info@wilmardenouden.nl>
2019-08-16 08:38:49 +00:00
wilmardo
30fb7de118 feat: Implement helm lint
Signed-off-by: wilmardo <info@wilmardenouden.nl>
2019-08-16 07:38:33 +00:00
Daniel-Pivonka
d621a58207 prometheus liveness probe sidecar
Signed-off-by: Daniel-Pivonka dpivonka@redhat.com
2019-08-13 17:51:41 +00:00
wilmardo
cba6115e30 Fix 1.13 charts
Signed-off-by: wilmardo <info@wilmardenouden.nl>
2019-08-13 16:42:15 +00:00
wilmardo
ca5fbc180c Rework of helm charts
Signed-off-by: wilmardo <info@wilmardenouden.nl>
2019-08-13 16:42:15 +00:00
Niels de Vos
31648c8feb provisioners: add reconfiguring of PID limit
The container runtime CRI-O limits the number of PIDs to 1024 by
default. When many PVCs are requested at the same time, it is possible
for the provisioner to start too many threads (or go routines) and
executing 'rbd' commands can start to fail. In case a go routine can not
get started, the process panics.

The PID limit can be changed by passing an argument to kubelet, but this
will affect all pids running on a host. Changing the parameters to
kubelet is also not a very elegant solution.

Instead, the provisioner pod can change the configuration itself. The
pod is running in privileged mode and can write to /sys/fs/cgroup where
the limit is configured.

With this change, the limit is configured to 'max', just as if there is
no limit at all. The logs of the csi-rbdplugin in the provisioner pod
will reflect the change it makes when starting the service:

    $ oc -n rook-ceph logs -c csi-rbdplugin csi-rbdplugin-provisioner-0
    ..
    I0726 13:59:19.737678       1 cephcsi.go:127] Initial PID limit is set to 1024
    I0726 13:59:19.737746       1 cephcsi.go:136] Reconfigured PID limit to -1 (max)
    ..

It is possible to pass a different limit on the commandline of the
cephcsi executable. The following flag has been added:

    --pidlimit=<int>       the PID limit to configure through cgroups

This accepts special values -1 (max) and 0 (default, do not
reconfigure). Other integers will be the limit that gets configured in
cgroups.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2019-08-13 14:43:29 +00:00
ShyamsundarR
44f7b1fe4b Use "rbd device list" to list and find rbd images and their device paths
This change also starts mapping nbd based access using ther rbd CLI
as, it is a prerequisite to get device listing for nbd as well.

Signed-off-by: ShyamsundarR <srangana@redhat.com>
2019-08-13 14:07:52 +00:00
Madhu Rajanna
02bcb5f16a Enable leader election in v1.14+
Use Deployment with leader election instead of StatefulSet

Deployment behaves better when a node gets disconnected
from the rest of the cluster - new provisioner leader
is elected in ~15 seconds, while it may take up to
5 minutes for StatefulSet to start a new replica.

Refer: kubernetes-csi/external-provisioner@52d1fbc

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-08-05 07:11:44 +00:00
ShyamsundarR
bd204d7d45 Use --keyfile option to pass keys to all Ceph CLIs
Every Ceph CLI that is invoked at present passes the key via the
--key option, and hence is exposed to key being displayed on
the host using a ps command or such means.

This commit addresses this issue by stashing the key in a tmp
file, which is again created on a tmpfs (or empty dir backed by
memory). Further using such tmp files as arguments to the --keyfile
option for every CLI that is invoked.

This prevents the key from being visible as part of the argument list
of the invoked program on the system.

Fixes: #318

Signed-off-by: ShyamsundarR <srangana@redhat.com>
2019-07-25 12:46:15 +00:00
Madhu Rajanna
f4c80dec9a Implement NodeStage and NodeUnstage for rbd
in NodeStage RPC call  we  have to map the
device to the node plugin and make  sure  the
the device will be mounted to  the global path

in  nodeUnstage request unmount the device from
global path and unmap the device

if the volume mode is block  we will be creating
a file inside a stageTargetPath  and it will be
considered  as the global path

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-07-24 12:49:21 +00:00
ShyamsundarR
c4a3675cec Move locks to more granular locking than CPU count based
As detailed in issue #279, current lock scheme has hash
buckets that are count of CPUs. This causes a lot of contention
when parallel requests are made to the CSI plugin. To reduce
lock contention, this commit introduces granular locks per
identifier.

The commit also changes the timeout for gRPC requests to Create
and Delete volumes, as the current timeout is 10s (kubernetes
documentation says 15s but code defaults are 10s). A virtual
setup takes about 12-15s to complete a request at times, that leads
to unwanted retries of the same request, hence the increased
timeout to enable operation completion with minimal retries.

Tests to create PVCs before and after these changes look like so,

Before:
Default master code + sidecar provisioner --timeout option set
to 30 seconds

20 PVCs
Creation: 3 runs, 396/391/400 seconds
Deletion: 3 runs, 218/271/118 seconds
  - Once was stalled for more than 8 minutes and cancelled the run

After:
Current commit + sidecar provisioner --timeout option set to 30 sec
20 PVCs
Creation: 3 runs, 42/59/65 seconds
Deletion: 3 runs, 32/32/31 seconds

Fixes: #279
Signed-off-by: ShyamsundarR <srangana@redhat.com>
2019-07-01 14:10:14 +00:00
Humble Chirammal
027331c186 Use sidecar which support cloning
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2019-06-28 01:11:06 +00:00
Madhu Rajanna
59d3365d3b update statefulset and daemonset api-version
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-06-25 14:00:46 +00:00
Madhu Rajanna
983f28ad2f Revert "Use Deployment with leader election instead of StatefulSet"
This reverts commit a151bec94b.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-06-14 13:39:03 +00:00
Madhu Rajanna
bccfafdfb2 update helm chart version
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-06-10 16:54:05 +05:30
Madhu Rajanna
a151bec94b Use Deployment with leader election instead of StatefulSet
Deployment behaves better when a node gets disconnected from the rest of
the cluster - new provisioner leader is elected in ~15 seconds, while
it may take up to 5 minutes for StatefulSet to start a new replica.

Refer: 52d1fbcf9d

Fixes: #335

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-06-10 09:51:22 +05:30
Humble Chirammal
45ae1c56e4 Promote sidecars to latest available version tags.
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2019-06-02 15:01:34 +05:30
Madhu Rajanna
2d560ba087 update ceph-csi to build and use a single docker image
currently, we have 3 docker files(cephcsi,rbd,cephfs) in the ceph-csi repo.
[commit ](85e121ebfe)
added by John to build a single image which can act as rbd or
cephfs based on the input configuration.

This PR updates the makefile and kubernetes templates to use
the unified image and also its deletes the other two dockerfiles.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-05-28 18:10:22 +00:00
ShyamsundarR
d02e50aa9b Removed config maps and replaced with rados omaps
Existing config maps are now replaced with rados omaps that help
store information regarding the requested volume names and the rbd
image names backing the same.

Further to detect cluster, pool and which image a volume ID refers
to, changes to volume ID encoding has been done as per provided
design specification in the stateless ceph-csi proposal.

Additional changes and updates,
- Updated documentation
- Updated manifests
- Updated Helm chart
- Addressed a few csi-test failures

Signed-off-by: ShyamsundarR <srangana@redhat.com>
2019-05-19 12:29:33 +00:00
Humble Chirammal
68ff602391 Resolve merge conflict
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2019-05-07 15:27:34 +05:30
Humble Chirammal
1eff2e1490 Merge branch 'master' of http://github.com/ceph/ceph-csi into csi-v1.0
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2019-05-07 15:14:14 +05:30
Miao Zhou
00e7e29996 bump up the chart version 2019-05-06 15:52:45 +08:00
Zhou Miao
a01c01b01b fix helm value pullPolicy mismatch bug 2019-04-25 12:03:44 +08:00
Kaushal M
63d00afb28
deploy: Use aggregated ClusterRoles
The kubernetes manifests and Helm templates have been updated to use
aggregated ClusterRoles. The same change has been done in Rook as well.

Refer rook/rook#2634 and rook/rook#2975

Signed-off-by: Kaushal M <kshlmster@gmail.com>
2019-04-17 11:15:08 +05:30
Yuxiang Zhu
35c55aeb68 add missing PV update permission for rbd attacher
PR #290 missed the update permission to persistentvolumes.

Without that permission, you will get the following error when attaching a RBD volume to a pod:

```
Warning  FailedAttachVolume  100s (x11 over 7m52s)  attachdetach-controller  AttachVolume.Attach failed for volume "pvc-d23f8745-60bb-11e9-bd35-5254001c78d6" : could not add PersistentVolume finalizer: persistentvolumes "pvc-d23f8745-60bb-11e9-bd35-5254001c78d6" is forbidden: User "system:serviceaccount:kube-system:rbd-csi-provisioner" cannot update resource "persistentvolumes" in API group "" at the cluster scope
```
2019-04-17 11:16:43 +08:00
Madhu Rajanna
849de000f4 updated helm chat version
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-04-04 11:13:16 +05:30
Madhu Rajanna
3767375b6a Add csidriver CRD
if attacher is not enabled, we need to
create the csidriver CRD with spec
to make attachRequired as false to
skip volume attach check in kube.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-04-04 11:11:29 +05:30
Madhu Rajanna
e4d830a2c2 remove extra node rules in provisioner
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-04-04 11:11:29 +05:30
Madhu Rajanna
c6b4e47723 add if condition for attacher
adding the condition will help us
to easily remove the attacher later.
or even we can add else condition
if we have an alternate to attacher.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-04-04 11:11:29 +05:30
Madhu Rajanna
54d52bb411 update attacher endpoint
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-04-04 11:11:29 +05:30
Madhu Rajanna
5c600a1bc5 update rbd helm chats to deploy attacher as sidecar container in provisioner pod
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-04-04 11:11:29 +05:30
Madhu Rajanna
3ef11e06c3 deploy attacher sidecar in rbd provisioner sts
currently we are deploying external-attacher
as a seperate statefulset, which leads to
attacher communicating with the node provisoner
daemonset, This PR deploys external-attacher
as a sidecar container inside provisioner
statefulset, so that external-provisioner
always communicates with the plugin responsible
for the provision controller capcabilities.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-04-04 11:11:29 +05:30
王怀宗
acdc759029 bump up the chart version 2019-04-01 16:48:30 +00:00
王怀宗
4228ceb51e rbd deploy csi-provisioner rbac add resources nodes get, list, watch #293 2019-04-01 16:48:30 +00:00
ShyamsundarR
2064e674a4 Addressed using k8s client APIs to fetch secrets
Based on the review comments addressed the following,
- Moved away from having to update the pod with volumes
when a new Ceph cluster is added for provisioning via the
CSI driver

- The above now used k8s APIs to fetch secrets
  - TBD: Need to add a watch mechanisim such that these
secrets can be cached and updated when changed

- Folded the Cephc configuration and ID/key config map
and secrets into a single secret

- Provided the ability to read the same config via mapped
or created files within the pod

Tests:
- Ran PV creation/deletion/attach/use using new scheme
StorageClass
- Ran PV creation/deletion/attach/use using older scheme
to ensure nothing is broken
- Did not execute snapshot related tests

Signed-off-by: ShyamsundarR <srangana@redhat.com>
2019-03-26 16:19:24 +00:00
Madhu Rajanna
52397b4dc4 rename socket directory to a common name
as the socket directory will be created
inside the container no need to follow
the plugin name in for the directory
creation, this will also reduce the code
changes if we want to change driver name.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-03-22 09:58:21 +05:30
Madhu Rajanna
497411b26c update readme to delete namespace
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-03-13 12:04:30 +05:30
Madhu Rajanna
d61a87b42e Fix driver name as per CSI spec
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-03-13 12:04:30 +05:30
Madhu Rajanna
c0745486a7 add event rules for provisioner
Fixes: #https://github.com/ceph/ceph-csi/pull/234#issuecomment-468967752

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-03-04 14:34:14 +00:00
Madhu Rajanna
eb14742874 bump helm chat version from 0.4.0 to 0.5.0
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-02-28 13:41:11 +05:30
Madhu Rajanna
2ab1f3e82d add csinodeinfos rules
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-02-27 19:32:07 +05:30
Madhu Rajanna
f4a0726226 Fix rbac issue in rbd plugin
remove unwanted rules and update
rbac to have permission to modify
endpoints and configmaps in the
current namespace.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-02-27 16:45:34 +05:30
Madhu Rajanna
3ac5af6ce6 Fix formatting in helm rbd readme
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-02-21 10:38:25 +05:30
Kevin Fox
e000dff1ea Fix identified issues
Signed-off-by: Kevin Fox <Kevin.Fox@pnnl.gov>
2019-02-08 12:16:30 +00:00
Kevin Fox
10af7e357d Add readme and notes to the chart
This adds a basic readme file and notes to the helm chart

Signed-off-by: Kevin Fox <Kevin.Fox@pnnl.gov>
2019-02-08 12:16:30 +00:00
Madhu Rajanna
cee9c4f8b2 Fix yamllint issues
Signed-off-by: Madhu Rajanna <mrajanna@redhat.com>
2019-02-07 12:19:14 +00:00
Kevin Fox
f172f089cc Update helm chart to match static files
This updates the helm chart to match all the changes that have
been made to the static manifest files.

Signed-off-by: Kevin Fox <Kevin.Fox@pnnl.gov>
2019-02-06 12:35:33 +00:00
Madhu Rajanna
9ba501617d update sidecar images to stable version (v1.0.1)
Fixes: #157

Signed-off-by: Madhu Rajanna <mrajanna@redhat.com>
2019-02-04 15:37:28 +05:30