Addressed using k8s client APIs to fetch secrets

Based on the review comments addressed the following,
- Moved away from having to update the pod with volumes
when a new Ceph cluster is added for provisioning via the
CSI driver

- The above now used k8s APIs to fetch secrets
  - TBD: Need to add a watch mechanisim such that these
secrets can be cached and updated when changed

- Folded the Cephc configuration and ID/key config map
and secrets into a single secret

- Provided the ability to read the same config via mapped
or created files within the pod

Tests:
- Ran PV creation/deletion/attach/use using new scheme
StorageClass
- Ran PV creation/deletion/attach/use using older scheme
to ensure nothing is broken
- Did not execute snapshot related tests

Signed-off-by: ShyamsundarR <srangana@redhat.com>

examples/rbd/template-ceph-cluster-ID-secret.yaml

@@ -0,0 +1,37 @@
+---
+# This is a template secret that helps define a Ceph cluster configuration
+# as required by the CSI driver. This is used when a StorageClass has the
+# "clusterID" defined as one of the parameters, to provide the CSI instance
+# Ceph cluster configuration information.
+apiVersion: v1
+kind: Secret
+metadata:
+  # The <cluster-fsid> is used by the CSI plugin to uniquely identify and use a
+  #   Ceph cluster, hence the value MUST match the output of the following
+  #   command.
+  #   - Output of: `ceph fsid`
+  name: ceph-cluster-<cluster-fsid>
+  namespace: default
+data:
+  # Base64 encoded and comma separated Ceph cluster monitor list
+  #   - Typically output of: `echo -n "mon1:port,mon2:port,..." | base64`
+  monitors: <BASE64-ENCODED-MONLIST>
+  # Base64 encoded and comma separated list of pool names from which volumes
+  # can be provisioned
+  pools: <BASE64-ENCODED-POOLIST>
+  # Base64 encoded admin ID to use for provisioning
+  #   - Typically output of: `echo -n "<admin-id>" | base64`
+  # Substitute the entire string including angle braces, with the base64 value
+  adminid: <BASE64-ENCODED-ID>
+  # Base64 encoded key of the provisioner admin ID
+  #   - Output of: `ceph auth get-key client.admin | base64`
+  # Substitute the entire string including angle braces, with the base64 value
+  adminkey: <BASE64-ENCODED-PASSWORD>
+  # Base64 encoded user ID to use for publishing
+  #   - Typically output of: `echo -n "<admin-id>" | base64`
+  # Substitute the entire string including angle braces, with the base64 value
+  userid: <BASE64-ENCODED-ID>
+  # Base64 encoded key of the publisher user ID
+  #   - Output of: `ceph auth get-key client.admin | base64`
+  # Substitute the entire string including angle braces, with the base64 value
+  userkey: <BASE64-ENCODED-PASSWORD>