mirror of
https://github.com/ceph/ceph-csi.git
synced 2025-06-13 02:33:34 +00:00
Addressed using k8s client APIs to fetch secrets
Based on the review comments addressed the following, - Moved away from having to update the pod with volumes when a new Ceph cluster is added for provisioning via the CSI driver - The above now used k8s APIs to fetch secrets - TBD: Need to add a watch mechanisim such that these secrets can be cached and updated when changed - Folded the Cephc configuration and ID/key config map and secrets into a single secret - Provided the ability to read the same config via mapped or created files within the pod Tests: - Ran PV creation/deletion/attach/use using new scheme StorageClass - Ran PV creation/deletion/attach/use using older scheme to ensure nothing is broken - Did not execute snapshot related tests Signed-off-by: ShyamsundarR <srangana@redhat.com>
This commit is contained in:
ShyamsundarR
committed by
mergify[bot]
mergify[bot]
parent
97f8c4b677
commit
2064e674a4
@ -14,7 +14,7 @@ Please consult the documentation for info about available parameters.
|
||||
|
||||
**NOTE:** See section
|
||||
[Cluster ID based configuration](#cluster-id-based-configuration) if using
|
||||
the `clusterID` instead of `monitors` or `monValueFromSecret` options in the
|
||||
the `clusterID` instead of `monitors` or `monValueFromSecret` option in the
|
||||
storage class for RBD based provisioning before proceeding.
|
||||
|
||||
After configuring the secrets, monitors, etc. you can deploy a
|
||||
@ -222,30 +222,29 @@ I/O size (minimum/optimal): 4194304 bytes / 4194304 bytes
|
||||
## Cluster ID based configuration
|
||||
|
||||
Before creating a storage class that uses the option `clusterID` to refer to a
|
||||
Ceph cluster,
|
||||
Ceph cluster, the following actions need to be completed.
|
||||
|
||||
**NOTE**: Substitute the output of `ceph fsid` instead of `<cluster-fsid>` in
|
||||
the mentioned template YAML files, and also the Ceph admin ID and
|
||||
credentials in their respective options. Further, update options like
|
||||
`monitors` and `pools` in the respective YAML files to contain the
|
||||
appropriate information.
|
||||
Get the following information from the Ceph cluster,
|
||||
|
||||
Create the following config maps and secrets
|
||||
* Ceph Cluster fsid
|
||||
* Output of `ceph fsid`
|
||||
* Used to substitute `<cluster-fsid>` references in the files below
|
||||
* Admin ID and key, that has privileges to perform CRUD operations on the Ceph
|
||||
cluster and pools of choice
|
||||
* Key is typically the output of, `ceph auth get-key client.admin` where
|
||||
`admin` is the Admin ID
|
||||
* Used to substitute admin/user id and key values in the files below
|
||||
* Ceph monitor list
|
||||
* Typically in the output of `ceph mon dump`
|
||||
* Used to prepare comma separated MON list where required in the files below
|
||||
|
||||
* `kubectl create -f ./rbd/template-ceph-cluster-ID-provisioner-secret.yaml`
|
||||
* `kubectl create -f ./rbd/template-ceph-cluster-ID-publish-secret.yaml`
|
||||
* `kubectl create -f ./rbd/template-ceph-cluster-ID-config.yaml`
|
||||
Update the template `rbd/template-ceph-cluster-ID-secret.yaml` with values from
|
||||
a Ceph cluster and create the following secret,
|
||||
|
||||
Modify the deployed CSI pods to additionally pass in the config maps and
|
||||
secrets as volumes,
|
||||
* `kubectl create -f rbd/template-ceph-cluster-ID-secret.yaml`
|
||||
|
||||
* `kubectl patch daemonset csi-rbdplugin --patch "$(cat ./rbd/template-csi-rbdplugin-patch.yaml)"`
|
||||
* `kubectl patch statefulset csi-rbdplugin-provisioner --patch "$(cat ./rbd/template-csi-rbdplugin-provisioner-patch.yaml)"`
|
||||
|
||||
Restart the provisioner and node plugin daemonset.
|
||||
|
||||
Storage class and snapshot class, using the `<cluster-fsid>` as the value for
|
||||
the option `clusterID`, can now be created on the cluster.
|
||||
Storage class and snapshot class, using `<cluster-fsid>` as the value for the
|
||||
option `clusterID`, can now be created on the cluster.
|
||||
|
||||
Remaining steps to test functionality remains the same as mentioned in the
|
||||
sections above.
|
||||
|
||||
@ -11,6 +11,9 @@ parameters:
|
||||
# OR,
|
||||
# Ceph cluster fsid, of the cluster to provision storage from
|
||||
# clusterID: <ceph-fsid>
|
||||
# If using clusterID based configuration, CSI pods need to be passed in a
|
||||
# secret named ceph-cluster-<cluster-fsid> that contains the cluster
|
||||
# information. (as in the provided template-ceph-cluster-ID-secret.yaml)
|
||||
# OR,
|
||||
# if "monitors" parameter is not set, driver to get monitors from same
|
||||
# secret as admin/user credentials. "monValueFromSecret" provides the
|
||||
@ -28,12 +31,18 @@ parameters:
|
||||
imageFeatures: layering
|
||||
|
||||
# The secrets have to contain Ceph admin credentials.
|
||||
# NOTE: If using "clusterID" instead of "monitors" above, the following
|
||||
# secrets MAY be added to the ceph-cluster-<cluster-fsid> secret and skipped
|
||||
# here
|
||||
csi.storage.k8s.io/provisioner-secret-name: csi-rbd-secret
|
||||
csi.storage.k8s.io/provisioner-secret-namespace: default
|
||||
csi.storage.k8s.io/node-publish-secret-name: csi-rbd-secret
|
||||
csi.storage.k8s.io/node-publish-secret-namespace: default
|
||||
|
||||
# Ceph users for operating RBD
|
||||
# NOTE: If using "clusterID" instead of "monitors" above, the following
|
||||
# IDs MAY be added to the ceph-cluster-<cluster-fsid> secret and skipped
|
||||
# here
|
||||
adminid: admin
|
||||
userid: kubernetes
|
||||
# uncomment the following to use rbd-nbd as mounter on supported nodes
|
||||
|
||||
@ -1,22 +0,0 @@
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: ceph-cluster-<cluster-fsid>
|
||||
namespace: default
|
||||
data:
|
||||
cluster-config: |
|
||||
{
|
||||
"version": 1,
|
||||
"cluster-config": {
|
||||
"cluster-fsid": "<ceph-fsid>",
|
||||
"monitors": [
|
||||
"<IP/DNS:port>",
|
||||
"<IP/DNS:port>"
|
||||
],
|
||||
"pools": [
|
||||
"<pool-name>",
|
||||
"<pool-name>"
|
||||
]
|
||||
}
|
||||
}
|
||||
@ -1,19 +0,0 @@
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
# The <cluster-fsid> is used by the CSI plugin to uniquely identify and use a
|
||||
# Ceph cluster, hence the value MUST match the output of the following
|
||||
# command.
|
||||
# - Output of: `ceph fsid`
|
||||
name: ceph-cluster-<cluster-fsid>-provisioner-secret
|
||||
namespace: default
|
||||
data:
|
||||
# Base64 encoded ID of the admin name
|
||||
# - Typically output of: `echo -n "<admin-id>" | base64`
|
||||
# Substitute the entire string including angle braces, with the base64 value
|
||||
subjectid: <BASE64-ENCODED-ID>
|
||||
# Credentials of the above admin/user
|
||||
# - Output of: `ceph auth get-key client.admin | base64`
|
||||
# Substitute the entire string including angle braces, with the base64 value
|
||||
credentials: <BASE64-ENCODED-PASSWORD>
|
||||
@ -1,19 +0,0 @@
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
# The <cluster-fsid> is used by the CSI plugin to uniquely identify and use a
|
||||
# Ceph cluster, hence the value MUST match the output of the following
|
||||
# command.
|
||||
# - Output of: `ceph fsid`
|
||||
name: ceph-cluster-<cluster-fsid>-publish-secret
|
||||
namespace: default
|
||||
data:
|
||||
# Base64 encoded ID of the admin name
|
||||
# - Typically output of: `echo -n "<admin-id>" | base64`
|
||||
# Substitute the entire string including angle braces, with the base64 value
|
||||
subjectid: <BASE64-ENCODED-ID>
|
||||
# Credentials of the above admin/user
|
||||
# - Output of: `ceph auth get-key client.admin | base64`
|
||||
# Substitute the entire string including angle braces, with the base64 value
|
||||
credentials: <BASE64-ENCODED-PASSWORD>
|
||||
37
examples/rbd/template-ceph-cluster-ID-secret.yaml
Normal file
37
examples/rbd/template-ceph-cluster-ID-secret.yaml
Normal file
@ -0,0 +1,37 @@
|
||||
---
|
||||
# This is a template secret that helps define a Ceph cluster configuration
|
||||
# as required by the CSI driver. This is used when a StorageClass has the
|
||||
# "clusterID" defined as one of the parameters, to provide the CSI instance
|
||||
# Ceph cluster configuration information.
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
# The <cluster-fsid> is used by the CSI plugin to uniquely identify and use a
|
||||
# Ceph cluster, hence the value MUST match the output of the following
|
||||
# command.
|
||||
# - Output of: `ceph fsid`
|
||||
name: ceph-cluster-<cluster-fsid>
|
||||
namespace: default
|
||||
data:
|
||||
# Base64 encoded and comma separated Ceph cluster monitor list
|
||||
# - Typically output of: `echo -n "mon1:port,mon2:port,..." | base64`
|
||||
monitors: <BASE64-ENCODED-MONLIST>
|
||||
# Base64 encoded and comma separated list of pool names from which volumes
|
||||
# can be provisioned
|
||||
pools: <BASE64-ENCODED-POOLIST>
|
||||
# Base64 encoded admin ID to use for provisioning
|
||||
# - Typically output of: `echo -n "<admin-id>" | base64`
|
||||
# Substitute the entire string including angle braces, with the base64 value
|
||||
adminid: <BASE64-ENCODED-ID>
|
||||
# Base64 encoded key of the provisioner admin ID
|
||||
# - Output of: `ceph auth get-key client.admin | base64`
|
||||
# Substitute the entire string including angle braces, with the base64 value
|
||||
adminkey: <BASE64-ENCODED-PASSWORD>
|
||||
# Base64 encoded user ID to use for publishing
|
||||
# - Typically output of: `echo -n "<admin-id>" | base64`
|
||||
# Substitute the entire string including angle braces, with the base64 value
|
||||
userid: <BASE64-ENCODED-ID>
|
||||
# Base64 encoded key of the publisher user ID
|
||||
# - Output of: `ceph auth get-key client.admin | base64`
|
||||
# Substitute the entire string including angle braces, with the base64 value
|
||||
userkey: <BASE64-ENCODED-PASSWORD>
|
||||
@ -12,22 +12,10 @@ spec:
|
||||
containers:
|
||||
- name: csi-rbdplugin
|
||||
volumeMounts:
|
||||
- name: provisioner-secret-<cluster-fsid>
|
||||
mountPath: "/etc/ceph-cluster-<cluster-fsid>-provisioner-secret"
|
||||
readOnly: true
|
||||
- name: publish-secret-<cluster-fsid>
|
||||
mountPath: "/etc/ceph-cluster-<cluster-fsid>-publish-secret"
|
||||
readOnly: true
|
||||
- name: ceph-cluster-<cluster-fsid>
|
||||
mountPath: "/etc/ceph-cluster-<cluster-fsid>/"
|
||||
mountPath: "/etc/csi-config/ceph-cluster-<cluster-fsid>"
|
||||
readOnly: true
|
||||
volumes:
|
||||
- name: provisioner-secret-<cluster-fsid>
|
||||
secret:
|
||||
secretName: ceph-cluster-<cluster-fsid>-provisioner-secret
|
||||
- name: publish-secret-<cluster-fsid>
|
||||
secret:
|
||||
secretName: ceph-cluster-<cluster-fsid>-publish-secret
|
||||
- name: ceph-cluster-<cluster-fsid>
|
||||
configMap:
|
||||
name: ceph-cluster-<cluster-fsid>
|
||||
secret:
|
||||
secretName: ceph-cluster-<cluster-fsid>
|
||||
|
||||
@ -12,22 +12,10 @@ spec:
|
||||
containers:
|
||||
- name: csi-rbdplugin
|
||||
volumeMounts:
|
||||
- name: provisioner-secret-<cluster-fsid>
|
||||
mountPath: "/etc/ceph-cluster-<cluster-fsid>-provisioner-secret"
|
||||
readOnly: true
|
||||
- name: publish-secret-<cluster-fsid>
|
||||
mountPath: "/etc/ceph-cluster-<cluster-fsid>-publish-secret"
|
||||
readOnly: true
|
||||
- name: ceph-cluster-<cluster-fsid>
|
||||
mountPath: "/etc/ceph-cluster-<cluster-fsid>/"
|
||||
mountPath: "/etc/csi-config/ceph-cluster-<cluster-fsid>"
|
||||
readOnly: true
|
||||
volumes:
|
||||
- name: provisioner-secret-<cluster-fsid>
|
||||
secret:
|
||||
secretName: ceph-cluster-<cluster-fsid>-provisioner-secret
|
||||
- name: publish-secret-<cluster-fsid>
|
||||
secret:
|
||||
secretName: ceph-cluster-<cluster-fsid>-publish-secret
|
||||
- name: ceph-cluster-<cluster-fsid>
|
||||
configMap:
|
||||
name: ceph-cluster-<cluster-fsid>
|
||||
secret:
|
||||
secretName: ceph-cluster-<cluster-fsid>
|
||||
|
||||
Reference in New Issue
Block a user