ceph-csi/deploy/rbd/kubernetes/csi-rbdplugin-provisioner.yaml

195 lines
5.7 KiB
YAML
Raw Normal View History

---
kind: Service
apiVersion: v1
metadata:
name: csi-rbdplugin-provisioner
labels:
app: csi-metrics
spec:
selector:
app: csi-rbdplugin-provisioner
ports:
- name: http-metrics
port: 8080
protocol: TCP
targetPort: 8680
---
kind: Deployment
apiVersion: apps/v1
metadata:
name: csi-rbdplugin-provisioner
spec:
replicas: 3
selector:
matchLabels:
app: csi-rbdplugin-provisioner
2018-07-18 14:48:43 +00:00
template:
metadata:
labels:
app: csi-rbdplugin-provisioner
spec:
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: app
operator: In
values:
- csi-rbdplugin-provisioner
topologyKey: "kubernetes.io/hostname"
serviceAccount: rbd-csi-provisioner
2018-07-18 14:48:43 +00:00
containers:
- name: csi-provisioner
image: quay.io/k8scsi/csi-provisioner:v1.6.0
2018-07-18 14:48:43 +00:00
args:
- "--csi-address=$(ADDRESS)"
- "--v=5"
- "--timeout=150s"
- "--retry-interval-start=500ms"
- "--enable-leader-election=true"
- "--leader-election-type=leases"
- "--feature-gates=Topology=true"
2018-07-18 14:48:43 +00:00
env:
- name: ADDRESS
value: unix:///csi/csi-provisioner.sock
2018-07-18 14:48:43 +00:00
imagePullPolicy: "IfNotPresent"
volumeMounts:
- name: socket-dir
mountPath: /csi
- name: csi-snapshotter
image: quay.io/k8scsi/csi-snapshotter:v2.1.0
args:
- "--csi-address=$(ADDRESS)"
- "--v=5"
- "--timeout=150s"
- "--leader-election=true"
env:
- name: ADDRESS
value: unix:///csi/csi-provisioner.sock
imagePullPolicy: "IfNotPresent"
securityContext:
privileged: true
volumeMounts:
- name: socket-dir
mountPath: /csi
- name: csi-attacher
image: quay.io/k8scsi/csi-attacher:v2.1.1
args:
- "--v=5"
- "--csi-address=$(ADDRESS)"
- "--leader-election=true"
- "--retry-interval-start=500ms"
env:
- name: ADDRESS
value: /csi/csi-provisioner.sock
imagePullPolicy: "IfNotPresent"
volumeMounts:
- name: socket-dir
mountPath: /csi
- name: csi-resizer
image: quay.io/k8scsi/csi-resizer:v0.5.0
args:
- "--csi-address=$(ADDRESS)"
- "--v=5"
- "--csiTimeout=150s"
- "--leader-election"
- "--retry-interval-start=500ms"
env:
- name: ADDRESS
value: unix:///csi/csi-provisioner.sock
imagePullPolicy: "IfNotPresent"
volumeMounts:
- name: socket-dir
mountPath: /csi
- name: csi-rbdplugin
securityContext:
privileged: true
capabilities:
add: ["SYS_ADMIN"]
# for stable functionality replace canary with latest release version
image: quay.io/cephcsi/cephcsi:canary
args:
- "--nodeid=$(NODE_ID)"
- "--type=rbd"
- "--controllerserver=true"
- "--endpoint=$(CSI_ENDPOINT)"
- "--v=5"
- "--drivername=rbd.csi.ceph.com"
provisioners: add reconfiguring of PID limit The container runtime CRI-O limits the number of PIDs to 1024 by default. When many PVCs are requested at the same time, it is possible for the provisioner to start too many threads (or go routines) and executing 'rbd' commands can start to fail. In case a go routine can not get started, the process panics. The PID limit can be changed by passing an argument to kubelet, but this will affect all pids running on a host. Changing the parameters to kubelet is also not a very elegant solution. Instead, the provisioner pod can change the configuration itself. The pod is running in privileged mode and can write to /sys/fs/cgroup where the limit is configured. With this change, the limit is configured to 'max', just as if there is no limit at all. The logs of the csi-rbdplugin in the provisioner pod will reflect the change it makes when starting the service: $ oc -n rook-ceph logs -c csi-rbdplugin csi-rbdplugin-provisioner-0 .. I0726 13:59:19.737678 1 cephcsi.go:127] Initial PID limit is set to 1024 I0726 13:59:19.737746 1 cephcsi.go:136] Reconfigured PID limit to -1 (max) .. It is possible to pass a different limit on the commandline of the cephcsi executable. The following flag has been added: --pidlimit=<int> the PID limit to configure through cgroups This accepts special values -1 (max) and 0 (default, do not reconfigure). Other integers will be the limit that gets configured in cgroups. Signed-off-by: Niels de Vos <ndevos@redhat.com>
2019-07-26 12:36:43 +00:00
- "--pidlimit=-1"
- "--rbdhardmaxclonedepth=8"
- "--rbdsoftmaxclonedepth=4"
env:
- name: POD_IP
valueFrom:
fieldRef:
fieldPath: status.podIP
- name: NODE_ID
valueFrom:
fieldRef:
fieldPath: spec.nodeName
- name: CSI_ENDPOINT
value: unix:///csi/csi-provisioner.sock
imagePullPolicy: "IfNotPresent"
volumeMounts:
- name: socket-dir
mountPath: /csi
- mountPath: /dev
name: host-dev
- mountPath: /sys
name: host-sys
- mountPath: /lib/modules
name: lib-modules
readOnly: true
- name: ceph-csi-config
mountPath: /etc/ceph-csi-config/
- name: ceph-csi-encryption-kms-config
mountPath: /etc/ceph-csi-encryption-kms-config/
- name: keys-tmp-dir
mountPath: /tmp/csi/keys
- name: liveness-prometheus
image: quay.io/cephcsi/cephcsi:canary
args:
- "--type=liveness"
- "--endpoint=$(CSI_ENDPOINT)"
- "--metricsport=8680"
- "--metricspath=/metrics"
- "--polltime=60s"
- "--timeout=3s"
env:
- name: CSI_ENDPOINT
value: unix:///csi/csi-provisioner.sock
- name: POD_IP
valueFrom:
fieldRef:
fieldPath: status.podIP
volumeMounts:
- name: socket-dir
mountPath: /csi
imagePullPolicy: "IfNotPresent"
2018-07-18 14:48:43 +00:00
volumes:
- name: host-dev
hostPath:
path: /dev
- name: host-sys
hostPath:
path: /sys
- name: lib-modules
hostPath:
path: /lib/modules
2018-07-18 14:48:43 +00:00
- name: socket-dir
emptyDir: {
medium: "Memory"
}
- name: ceph-csi-config
configMap:
name: ceph-csi-config
- name: ceph-csi-encryption-kms-config
configMap:
name: ceph-csi-encryption-kms-config
- name: keys-tmp-dir
emptyDir: {
medium: "Memory"
}