Commit Graph

894 Commits

Author SHA1 Message Date
Madhu Rajanna
c9943320ac cephfs: skip NetNamespaceFilePath if the volume is pre-provisioned
In case of pre-provisioned volume the clusterID is
not set in the volume context as the clusterID is missing
we cannot extract the NetNamespaceFilePath from the
configuration file. For static volume and dynamically
provisioned volume the clusterID is set.

Note:- This is a special case to support mounting PV
without clusterID parameter.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-06-03 07:25:25 +00:00
Rakshith R
7688306f87 rbd: use vaultAuthPath variable name in error msg
Before the change, the error msg was the following:
```
failed to set VAULT_AUTH_MOUNT_PATH in Vault config: path is empty
```
`vaultAuthPath` is the actual variable name set by the
user. The error message will now be the following:
```
failed to set "vaultAuthPath" in vault config: path is empty
```

Signed-off-by: Rakshith R <rar@redhat.com>
2022-05-26 07:37:48 +00:00
Rakshith R
894c20f792 nfs: add support for pvc-pvc clone
This commit adds support for pvc-pvc clone.
Only capability needed to be advertised, the
underlying support is already provided by cephfs
backend.

Signed-off-by: Rakshith R <rar@redhat.com>
2022-05-24 18:13:02 +00:00
Rakshith R
24515b509f nfs: add support for create & delete snapshot
This commits adds support for creation and
deletion of nfs snapshots based on cephfs.

Signed-off-by: Rakshith R <rar@redhat.com>
2022-05-24 18:13:02 +00:00
Prasanna Kumar Kalever
6470cf3343 rbd: fix bug handling GetKrbdSupportedFeatures()
continue running rbd driver when /sys/bus/rbd/supported_features file is
missing, do not bailout.

Fixes: #2678
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2022-05-15 15:10:08 +00:00
Prasanna Kumar Kalever
83cc1b0e58 rbd: handle when krbdFeatures is zero
krbdFeatures is set to zero when kernel version < 3.8, i.e. in  case where
/sys/bus/rbd/supported_features is absent and we are unable to prepare
the krbd attributes based on kernel version.

When krbdFeatures is set to zero fallback to NBD only when autofallback
is turned ON.

Fixes: #2678
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2022-05-15 15:10:08 +00:00
Prasanna Kumar Kalever
e53fd87154 rbd: prepare krbd feature attrs if supported_features file is absent
Upstream /sys/bus/rbd/supported_features is part of Linux kernel v4.11.0
Prepare the attributes and use them in case if
/sys/bus/rbd/supported_features is missing.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2022-05-15 15:10:08 +00:00
Prasanna Kumar Kalever
27f503c144 rbd: unset parent PVC metadata on CreateVolume From Volume
Unset the parent PVC metadata on the temp clone rbd image

Fixes: #2970
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2022-05-12 15:54:09 +00:00
Prasanna Kumar Kalever
e0f34a6d60 rbd: unset snapshot metadata on CreateVolume From snapshot
Unset the snapshot metadata from the rbd image created from the snapshot

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2022-05-12 15:54:09 +00:00
Prasanna Kumar Kalever
d89c5fb39f rbd: unset PVC metadata on CreateSnapshot
Unset the PVC metadata on the rbd image created for the snapshot

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2022-05-12 15:54:09 +00:00
Prasanna Kumar Kalever
bac33262ae rbd: add unset volume/snapshot metadata utility functions
Added
GetVolumeMetadataKeys()
GetSnaoshotMetadataKeys()
unsetVolumeMetadata() and
unsetSnapshotMetadata()

functions.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2022-05-12 15:54:09 +00:00
Prasanna Kumar Kalever
1fd5277b3c cleanup: simplify setVolumeMetadata and rename it
Move k8s.GetVolumeMetadata() out of setVolumeMetadata() and rename it to
setAllMetadata() so that the same can be used for setting volume and
snapshot metadata.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2022-05-12 15:54:09 +00:00
Niels de Vos
36e51402cb nfs: support ExpandVolume CSI procedure
There is not much the NFS-provisioner needs to do to expand a volume,
everything is handled by the CephFS components.

NFS does not need a resize on the node, so only ControllerExpandVolume
is required.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-05-10 17:43:59 +00:00
Madhu Rajanna
70674565df rbd: consider rbd as default mounter if not set
For the default mounter the mounter option
will not be set in the storageclass and as it is
not available in the storageclass same will not
be set in the volume context, Because of this the
mapOptions are getting discarded. If the mounter
is not set assuming it's an rbd mounter.

Note:- If the mounter is not set in the storageclass
we can set it in the volume context explicitly,
Doing this check-in node server to support backward
existing volumes and the check is minimal we are not
altering the volume context.

fixes: #3076

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-05-09 20:00:11 +00:00
Marcus Röder
a95a6213eb util: support systems using the new cgroup v2 structure
With cgroup v2, the location of the pids.max file changed and so did the
/proc/self/cgroup file

new /proc/self/cgroup file
`
0::/user.slice/user-500.slice/session-14.scope
`

old file:
`
11:pids:/user.slice/user-500.slice/session-2.scope
10:blkio:/user.slice
9:net_cls,net_prio:/
8:perf_event:/
...
`

There is no directory per subsystem (e.g. /sys/fs/cgroup/pids) any more, all
files are now in one directory.

fixes: https://github.com/ceph/ceph-csi/issues/3085

Signed-off-by: Marcus Röder <m.roeder@yieldlab.de>
2022-05-07 20:38:48 +00:00
Rakshith R
f1ccc4eced rbd: support pvc-pvc clone with different sc & encryption
This commit makes modification so as to allow pvc-pvc clone
with different storageclass having different encryption
configs.
This commit also modifies `copyEncryptionConfig()` to
include a `isEncrypted()` check within the function.

Signed-off-by: Rakshith R <rar@redhat.com>
2022-05-06 10:32:21 +00:00
Rakshith R
bd57feb26e rbd: use vaultAuthPath variable name in error msg
Before the change, the error msg was the following:
```
failed to set VAULT_AUTH_MOUNT_PATH in Vault config: path is empty
```
`vaultAuthPath` is the actual variable name set by the
user. The error message will now be the following:
```
failed to set "vaultAuthPath" in vault config: path is empty
```

Signed-off-by: Rakshith R <rar@redhat.com>
2022-05-05 05:49:31 +00:00
Niels de Vos
9d7faf850f nfs: delete the CephFS volume when the export is already removed
In case the NFS-export has already been removed from the NFS-server, but
the CSI Controller was restarted, a retry to remove the NFS-volume will
fail with an error like:

> GRPC error: ....: response status not empty: "Export does not exist"

When this error is reported, assume the NFS-export was already removed
from the NFS-server configuration, and continue with deleting the
backend volume.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-05-04 21:31:06 +00:00
Madhu Rajanna
d2bc9743f7 cephfs: add netNamespaceFilePath for CephFS
as same host directory is not shared between
the cephfs and the rbd plugin pod. we need
to keep the netNamespaceFilePath separately
for both cephfs and rbd. CephFS plugin will
use this path to execute mount -t commands.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-04-19 12:28:46 +00:00
Madhu Rajanna
eb4bfb7326 cleanup: use block comment for ClusterInfo example
Adjusted the mix of tabs and the spaces and also
used block comment for better readability.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-04-19 12:28:46 +00:00
Madhu Rajanna
b4acbd08a5 rbd: move radosNamespace to RBD section
As radosNamespace is more specific to
RBD not the general ceph configuration. Now
we introduced a new RBD section for RBD specific
options, Moving the radosNamespace to RBD section
and keeping the radosNamespace still under the
global ceph level configration for backward
compatibility.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-04-19 12:28:46 +00:00
Madhu Rajanna
766346868e util: Add RBD specific options in clusterInfo
As the netNamespaceFilePath can be separate for
both cephfs and rbd adding the netNamespaceFilePath
path for RBD, This will help us to keep RBD and
CephFS specific options separately.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-04-19 12:28:46 +00:00
Niels de Vos
2b71aac752 nfs: return gRPC status from CephFS CreateVolume failure
The NFS Controller returns a non-gRPC error in case the CreateVolume
call for the CephFS volume fails. It is better to return the gRPC-error
that the CephFS Controller passed along.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-04-19 08:23:16 +00:00
Humble Chirammal
fcd0f4713a cleanup: correct typos in test description and source code
this commit correct typos in various places.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-04-18 10:29:08 +00:00
Humble Chirammal
4c4879ba8b cleanup: remove import alias for fence library
this commit remove unneeded import alias of fence library
from the network_fence test.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-04-18 10:29:08 +00:00
Madhu Rajanna
c245436ec4 util: fix logging in ExecuteCommandWithNSEnter
log the nsenter and its argument after executing
the command with the nsenter CLI.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-04-14 12:17:21 +00:00
Niels de Vos
28369702d2 nfs: use go-ceph API for creating/deleting exports
Recent versions of Ceph allow calling the NFS-export management
functions over the go-ceph API.

This seems incompatible with older versions that have been tested with
the `ceph nfs` commands that this commit replaces.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-04-14 08:01:45 +00:00
Madhu Rajanna
d886ab0d66 rbd: use leases for leader election
use leases for leader election instead
of the deprecated configmap based leader
election.

This PR is making leases as default leader election
refer https://github.com/kubernetes-sigs/
controller-runtime/pull/1773, default from configmap
to configmap leases was done with
https://github.com/kubernetes-sigs/
controller-runtime/pull/1144.

Release notes https://github.com/kubernetes-sigs/
controller-runtime/releases/tag/v0.7.0

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-04-14 06:46:50 +00:00
Madhu Rajanna
64a9b1fa59 rbd: consider remote image health for primary
To consider the image is healthy during the Promote
operation currently we are checking only the image
state on the primary site. If the network is flaky
or the remote site is down the image health is
not as expected. To make sure the image is healthy
across the clusters check the state on both local
and the remote clusters.

some details:
https://bugzilla.redhat.com/show_bug.cgi?id=2014495

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-04-13 08:37:23 +00:00
Madhu Rajanna
dffb6e72c2 rbd: check nbd tool features only for rbd driver
calling setRbdNbdToolFeatures inside an init
gets called in main.go for both cephfs and rbd
driver. instead of calling it in init function
calling this in rbd driver.go as this is specific
to rbd.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-04-11 21:18:27 +00:00
Humble Chirammal
959df4dbac doc: correct typos in struct field comments and release.md
corrected strings in the release guide and util server.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-04-11 06:23:25 +00:00
Prasanna Kumar Kalever
41fe2c7dda rbd: set metadata on the snapshot
Set snapshot-name/snapshot-namespace/snapshotcontent-name details
on RBD backend snapshot image as metadata on snapshot

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2022-04-08 15:43:14 +00:00
Prasanna Kumar Kalever
0ef79c6fc0 rbd: set metadata on restart of provisioner pod
Make sure to set metadata when image exist, i.e. if the provisioner pod
is restarted while createVolume is in progress, say it created the image
but didn't yet set the metadata.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2022-04-08 15:43:14 +00:00
Prasanna Kumar Kalever
ae5925f04c rbd: update PV/PVC metadata on a reattach of PV
Example if a PVC was delete by setting `persistentVolumeReclaimPolicy` as
`Retain` on PV, and PV is reattached to a new PVC, we make sure to update
PV/PVC image metadata on a PV reattach.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2022-04-08 15:43:14 +00:00
Prasanna Kumar Kalever
0119d69ab2 rbd: set PV/PVC details on the image as metadata on create
This helps Monitoring solutions without access to Kubernetes clusters to
display the details of the PV/PVC/NameSpace in their dashboard.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2022-04-08 15:43:14 +00:00
Prasanna Kumar Kalever
4d750ed0e5 rbd: add set/Get VolumeMetadata() utility function
Define and use PV and PVC metadata keys used by external provisioner.
The CSI external-provisioner (v1.6.0+) introduces the
--extra-create-metadata flag, which automatically sets map<string, string>
parameters in the CSI CreateVolumeRequest.

Add utility functions to set/Get PV/PVC/PVCNamespace metadata on image

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2022-04-08 15:43:14 +00:00
Madhu Rajanna
7b2aef0d81 util: add support for the nsenter
add support to run rbd map and mount -t
commands with the nsenter.

complete design of pod/multus network
is added here https://github.com/rook/rook/
blob/master/design/ceph/multus-network.md#csi-pods

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-04-08 10:23:21 +00:00
Prasanna Kumar Kalever
d760d0ab6d rbd: check for cookie support from kernel
Currently we only check if the rbd-nbd tool supports cookie feature.
This change will also defend cookie addition based on kernel version

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2022-04-04 09:51:13 +00:00
Madhu Rajanna
f8bbd2f60f cephfs: fix omap deletion in DeleteSnapshot
The omap is stored with the requested
snapshot name not with the subvolume
snapshotname. This fix uses the correct
snapshot request name to cleanup the omap
once the subvolume snapshot is deleted.

fixes: #2974

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-03-31 13:46:03 +00:00
Niels de Vos
1da19680b4 nfs: support new and old NFS-management commands
The `ceph nfs export ...` commands have changed in recent Ceph releases.
Use the most recent command as a default, fall back to the older command
when an error is reported.

This shoud make the NFS-provisioner work on any current Ceph version.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-03-31 11:28:40 +00:00
Madhu Rajanna
f90408be4d rbd: increase force promote timeout to 2 minutes
Increase the timeout to 2 minutes to give enough time
for rollback to complete.
As rollback is performed by the force-promote command it,
at times, may take more than a minute
(based on dirty blocks that need to be rolled
back approximately) to rollback.

The added extra 1 minute is useful though to avoid
multiple calls to complete the rollback and in
extremely corner cases to avoid failures in the
first instance of the call when the mirror watcher
is not yet removed (post scaling down the
RBD mirror instance)

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-03-30 13:46:27 +00:00
Thibaut Blanchard
e874c9c11b rbd: fix topology snapshot pool
Restoring a snapshot with a new PVC results with a wrong
dataPoolName in case of initial volume linked
to a storageClass with topology constraints and erasure coding.

Signed-off-by: Thibaut Blanchard <thibaut.blanchard@gmail.com>
2022-03-30 04:40:30 +00:00
Niels de Vos
885295fcc9 nfs: store the NFS-cluster name in the journal
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-03-28 11:23:17 +00:00
Niels de Vos
3b4d193ca8 journal: add StoreAttribute/FetchAttribute
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-03-28 11:23:17 +00:00
Niels de Vos
010fd816dd nfs: store the calling Context in NFSVolume
NFSVolume instances are short lived, they only extist for a certain gRPC
procedure. It is easier to store the calling Context in the NFSVolume
struct, than to pass it to some of the functions that require it.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-03-28 11:23:17 +00:00
Niels de Vos
6d83df9cc9 nfs: add basic provisioner with create/delete procedures
These NFS Controller and Identity servers are the base for the new
provisioner. The functionality is currently extremely limited, follow-up
PRs will implement various CSI procedures.

CreateVolume is implemented with the bare minimum. This makes it
possible to create a volume, and mount it with the
kubernetes-csi/csi-driver-nfs NodePlugin.

DeleteVolume unexports the volume from the Ceph managed NFS-Ganesha
service. In case the Ceph cluster provides multiple NFS-Ganesha
deployments, things might not work as expected. This is going to be
addressed in follow-up improvements.

Lots of TODO comments need to be resolved before this can be declared
"production ready". Unit- and e2e-tests are missing as well.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-03-28 11:23:17 +00:00
Robert Vasek
f6ae612003 util: added reference tracker
RT, reference tracker, is key-based implementation of a reference counter.
Unlike an integer-based counter, RT counts references by tracking unique
keys. This allows accounting in situations where idempotency must be
preserved. It guarantees there will be no duplicit increments or decrements
of the counter.

Signed-off-by: Robert Vasek <robert.vasek@cern.ch>
2022-03-27 19:24:26 +00:00
Rakshith R
40de75e0db rbd: modify oidc token file path according to FHS 3.0
OIDC token file path has been modified from
`/var/run/secrets/token` to `/run/secrets/tokens`.
This has been done to ensure compliance with
FHS 3.0.

refer:
https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch05s13.html

Signed-off-by: Rakshith R <rar@redhat.com>
2022-03-23 13:29:35 +00:00
Madhu Rajanna
8c5e414d53 rbd: do not read pvc namespace from volume attributes
Below are the 3 different cases where we need
the PVC namespace for encryption

* CreateVolume:- Read the namespace from the
createVolume parameters and store it in the omap
* NodeStage:- Read the namespace from the omap
not from the volumeContext
* Regenerate:- Read the pvc namespace from the claimRef
not from the volumeAttributes.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-03-21 08:54:43 +00:00
Madhu Rajanna
77011fbc61 cephfs: remove kubernetes csi prefixed parameters
remove kubernetes csi prefixed parameters
from the volumeContext as we dont want
to store it in the PV VolumeAttributes.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-03-21 08:54:43 +00:00