1
0
mirror of https://github.com/ceph/ceph-csi.git synced 2024-12-22 13:00:19 +00:00
Commit Graph

1194 Commits

Author SHA1 Message Date
Praveen M
4d466843b9 cephfs: add read affinity mount option
This commit makes use of crush location labels from node
labels to supply `crush_location` and `read_from_replica=localize`
options during mount. Using these options, cephfs
will be able to redirect reads to the closest OSD,
improving performance.

Signed-off-by: Praveen M <m.praveen@ibm.com>
2023-11-22 13:13:01 +00:00
Praveen M
e9983306d9 util: remove metrics argument from NewMiddlewareServerOption
This commit removes metrics argument from NewMiddlewareServerOption
as it always set to false.

Signed-off-by: Praveen M <m.praveen@ibm.com>
2023-11-17 10:15:32 +00:00
Niels de Vos
0c13a70363 util: include request-IDs in all gRPC calls for the Controller
Snapshot procedures do not seem to contain the `Req-ID:` prefix in the
logs anymore (or weren't they there at all?) for some reason. This adds
them back.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2023-11-16 16:50:16 +00:00
Praveen M
fb8bb3f576 cephfs: support for kernelMountOptions & fuseMountOptions per cluster
Implemented the capability to include kernel mount options and
fuse mount options for individual clusters within the ceph-csi-config
ConfigMap.This allows users to configure the kernel/fuse mount options
for each cluster separately. The mount options specified in the ConfigMap
will supersede those provided via command line arguments.

Signed-off-by: Praveen M <m.praveen@ibm.com>
2023-11-15 11:43:27 +00:00
Praveen M
1e4ff115a7 util: added GetCephFSMountOptions method
This commit adds GetCephFSMountOptions util method which returns
KernelMountOptions and fuseMountOptions for cluster `clusterID`.

Signed-off-by: Praveen M <m.praveen@ibm.com>
2023-11-15 11:43:27 +00:00
Praveen M
6f9b1ab04d cephfs: remove subvolumegroup creation
Signed-off-by: Praveen M <m.praveen@ibm.com>
2023-11-10 11:41:35 +00:00
Praveen M
c4e373c72f deploy: support for read affinity options per cluster
Implemented the capability to include read affinity options
for individual clusters within the ceph-csi-config ConfigMap.
This allows users to configure the crush location for each
cluster separately. The read affinity options specified in
the ConfigMap will supersede those provided via command line arguments.

Signed-off-by: Praveen M <m.praveen@ibm.com>
2023-11-08 21:17:00 +00:00
Praveen M
93f2ed681f util: added RunsOnKubernetes() function
Signed-off-by: Praveen M <m.praveen@ibm.com>
2023-11-08 21:17:00 +00:00
Praveen M
9cec05f484 util: added read affinity related functions and unit testcases
This commit adds util functions related to read affinity
and unit testcases for the same.

Signed-off-by: Praveen M <m.praveen@ibm.com>
2023-11-08 21:17:00 +00:00
Praveen M
a93f3e24ba util: moved GetNodeLabels() under internal/util/k8s
Signed-off-by: Praveen M <m.praveen@ibm.com>
2023-11-08 21:17:00 +00:00
Madhu Rajanna
1b39b82a85 cephfs: avoid variable shadowing
Avoiding variable shadowing as it
becomes hard to debug this kind of
issues in real enviroment.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2023-11-08 19:25:51 +00:00
Madhu Rajanna
c25f572ae0 cephfs: remove subvolume during clone
If any operations like Resize, Deleting
snapshot fails, we need to remove
both snapshot and the clone to avoid
resource leak.

closes: 

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2023-11-08 19:25:51 +00:00
Riya Singhal
2598300f66 cephfs: add test case for ipv6 address
Signed-off-by: Riya Singhal <rsinghal@redhat.com>
2023-11-08 15:43:58 +00:00
Riya Singhal
0631c15025 cephfs: update fetchIP to support ipv6 addresses
Signed-off-by: Riya Singhal <rsinghal@redhat.com>
2023-11-08 15:43:58 +00:00
Madhu Rajanna
304462c7cc cleanup: fix spellcheck errors
fixed spellcheck errors caught in
CI.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2023-11-08 12:32:06 +00:00
Niels de Vos
76d9400725 cleanup: remove left-overs of ReplicationServer in csi-common
The ReplicationServer is not used anymore, the functionality has moved
to CSI-Addons and the `internal/csi-addons/rbd` package. These last
references were not activated anywhere, so can be removed without any
impact.

See-also: 
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2023-11-07 07:37:16 +00:00
Madhu Rajanna
9f753889ed rbd: remove deprecated rbdImageRequiresEncryption
remove support for deprecated
rbdImageRequiresEncryption case.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2023-11-04 08:14:51 +00:00
Madhu Rajanna
1e05a0da7c util: remove RadosNamespace from configmap
remove RadosNamespace from the configmap main
section as its already added to the rbd section

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2023-11-04 08:14:51 +00:00
Niels de Vos
4d3b1fc46f util: if FilesystemNodeGetVolumeStats succeeds the volume is healthy
When FilesystemNodeGetVolumeStats() succeeds, the volume must be
healthy. This can be included in the VolumeCondition CSI message by
default.

Checks that detect an abnormal VolumeCondition should prevent calling
FilesystemNodeGetVolumeStats() as it is possible that the function will
hang.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2023-11-03 13:41:44 +00:00
Niels de Vos
7d96cafad7 cephfs: enable VolumeCondition with new health-checker
The HealthChecker is configured to use the Staging path pf the volume,
with a `.csi/` subdirectory. In the future this directory could be a
directory that is not under the Published directory.

Fixes: 
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2023-11-03 13:41:44 +00:00
Niels de Vos
54fc65a561 util: add health-checker for periodic filesystem checks
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2023-11-03 13:41:44 +00:00
Madhu Rajanna
00d6971dbd util: rearrange struct to fix lint
re-arrange the struct members to
fix below lint issue

```
struct of size 336 bytes could be of size 328 bytes
```

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2023-11-03 10:04:50 +00:00
Madhu Rajanna
3ea540bf0f util: remove deprecated grpc metrics
This commit removes the deprecated
grpc related code from cephcsi.

fixes: 

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2023-11-03 10:04:50 +00:00
Praveen M
c09700b568 cleanup: removed unused SnapshotInfo from CheckSnapExists()
Signed-off-by: Praveen M <m.praveen@ibm.com>
2023-11-01 11:32:07 +00:00
Praveen M
ad3a3d704a cephfs: remove snapshot protect/unprotect
This commit eliminates the code for protecting and unprotecting
snapshots, as the functionality to protect and unprotect snapshots
is being deprecated.

Signed-off-by: Praveen M <m.praveen@ibm.com>
2023-11-01 11:32:07 +00:00
Riya Singhal
1fc9678342 cephfs: expose FenceControllerServer in driver.go to handle the requests
Signed-off-by: Riya Singhal <rsinghal@redhat.com>
2023-10-24 16:36:41 +00:00
Riya Singhal
74dfb7b913 cephfs: adds unit testing for unfencing procedure
Signed-off-by: Riya Singhal <rsinghal@redhat.com>
2023-10-24 16:36:41 +00:00
Riya Singhal
14b06837d0 cephfs: implement the logic for unfencing procedure
this commit un-blocklists the clients provided in cidr
for unfencing operation.

Signed-off-by: Riya Singhal <rsinghal@redhat.com>
2023-10-24 16:36:41 +00:00
Riya Singhal
1420ad193a cephfs: adds unit testing for fencing procedure
Signed-off-by: Riya Singhal <rsinghal@redhat.com>
2023-10-24 16:36:41 +00:00
Riya Singhal
d925937d53 cephfs: adding unit test for fetchID
Signed-off-by: Riya Singhal <rsinghal@redhat.com>
2023-10-24 16:36:41 +00:00
Riya Singhal
b8e74e62c0 cephfs: adding unit test for fetchIP for client eviction
Signed-off-by: Riya Singhal <rsinghal@redhat.com>
2023-10-24 16:36:41 +00:00
Riya Singhal
6b0c412996 cephfs: add network fence service to identity.go
Signed-off-by: Riya Singhal <rsinghal@redhat.com>
2023-10-24 16:36:41 +00:00
Riya Singhal
9d5ee285a2 cephfs: adds the implementation of client eviction
this commit adds client eviction to cephfs, based
on the IPs in cidr block, it evicts those IPs from
the network.

Signed-off-by: Riya Singhal <rsinghal@redhat.com>
2023-10-24 16:36:41 +00:00
Madhu Rajanna
b062479012 util: remove unimplemented methods
As we are using UnimplementedControllerServer
we dont need to implement mock
methods.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2023-10-23 07:02:40 +00:00
Madhu Rajanna
e5d2d3eeb4 util: add UnimplementedControllerServer to default
Add UnimplementedControllerServer to the default
implementation to avoid missing method
error.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2023-10-23 07:02:40 +00:00
Rakshith R
e46a007961 rebase: update github.com/libopenstorage/secrets to latest
With this update, we no longer import github.com/hashicorp/vault
which now is under BSL license.
https://github.com/hashicorp/vault/blob/main/LICENSE

resolves: 

Signed-off-by: Rakshith R <rar@redhat.com>
2023-10-17 11:06:24 +00:00
karthik-us
5ff0607360 e2e: To test 500MB PVC creation
Adding e2e test to check for successful PVC creation
of 500MB.

Signed-off-by: karthik-us <ksubrahm@redhat.com>
2023-10-12 17:31:46 +00:00
karthik-us
5a0eeb882e cephfs: Fix cephfs PVC sizing
Issue:
The RoundOffCephFSVolSize() function omits the fractional
part when calculating the size for cephfs volumes, leading
to the created volume capacity to be lesser than the requested
volume capacity.

Fix:
Consider the fractional part during the size calculation so the
rounded off volume size will be greater than or equal to the
requested volume size.

Signed-off-by: karthik-us <ksubrahm@redhat.com>
Fixes: 
2023-10-12 17:31:46 +00:00
Rakshith R
d516a1d66d cephfs: safeguard subVolumeGroupCreated map from race condition
Multiple go-routines may simultaneously create the
subVolumeGroupCreated map or  write into it
for a particular group.

This commit safeguards subVolumeGroupCreated map
from concurrent creation/writes while allowing for multiple
readers.

Signed-off-by: Rakshith R <rar@redhat.com>
2023-10-10 21:04:05 +00:00
Rakshith R
82f1323af4 cephfs: safeguard localClusterState struct from race conditions
Multiple go-routines may simultaneously check for a clusterID's
presence in clusterAdditionalInfo and create an entry if it is
absent. This set of operation needs to be serialized.

Therefore, this commit safeguards clusterAdditionalInfo map
from concurrent writes with a mutex to prevent the above problem.

Signed-off-by: Rakshith R <rar@redhat.com>
2023-10-10 21:04:05 +00:00
Praveen M
0ed7a64191 rbd: update snap RbdImageName in createSnapshot
This PR updates the snapshot RbdImageName in
`createSnapshot` method. This resolves the
incorrect statement logged during snapshot creation.

Signed-off-by: Praveen M <m.praveen@ibm.com>
2023-10-03 11:45:03 +00:00
Praveen M
e504987984 rbd: update snap RbdImageName
This commit updates the snapshot RbdImageName with the clone
RbdImageName before snapshot creation. This will fix the
incorrect log statement.

Signed-off-by: Praveen M <m.praveen@ibm.com>
2023-09-28 11:51:13 +00:00
HF
5411a69839 rbd: fixed all potential crashing when decoding volume ID failed
Signed-off-by: HF <crazytaxii666@gmail.com>
2023-09-06 13:46:22 +00:00
HF
80ad5b6b8f rbd: fixed csi-rbdplugin crashes when decoding volume ID failed
Signed-off-by: HF <crazytaxii666@gmail.com>
2023-09-05 12:08:53 +00:00
Madhu Rajanna
683821c407 rbd: discard not found error from GetMetadata
During ResyncVolume call, discard not found
error from GetMetadata API. If the image gets
resynced the local image creation time will be
lost, if the key is not present in the image
metadata then we can assume that the image is
already resynced.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2023-09-04 05:55:52 +00:00
Madhu Rajanna
771470d975 cephfs: add support for RWX clone from ROX
Add support to create RWX clone from the
ROX clone, in ceph no subvolume clone is
created when ROX clone is created from a
snapshot just a internal ref counter is
added. This PR allows creating a RWX clone
from a ROX clone which allows users to create
RW copy of PVC where cephcsi will identify
the snapshot created for the ROX volume and
creates a subvolume from the CephFS snapshot.

updates: 

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2023-08-31 11:51:18 +00:00
Madhu Rajanna
5d034b1124 cephfs: block creation of ROX clone from ROX volume
As there is no usecase currently, blocking the
creation of ROX clone from the ROX volume.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2023-08-31 11:51:18 +00:00
Madhu Rajanna
e013cfed15 rbd: fix resync issue
During the Demote volume store
the image creation timestamp.

During Resync do below operation

* Check image creation timestamp
stored during Demote operation
and current creation timestamp during Resync
and check both are equal and its for
force resync then issue resync
* If the image on both sides is
not in unknown state, check
last_snapshot_timestamp on the
local mirror description, if its present
send volumeReady as false or else return
error message.

If both the images are in up+unknown the
send volumeReady as true.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2023-08-30 09:07:46 +00:00
Madhu Rajanna
4016876c9d cephfs: fix make go-lint error
Not sure why but go-lint is failing
with below error and this fix is required
to make it pass

```
directive `//nolint:staticcheck // See comment above.`
is unused for linter "staticcheck" (nolintlint)
```

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2023-08-30 09:07:46 +00:00
karthik-us
855d09c1cd rebase: replace MetricsBindAddress with Metrics
MetricsBindAddress is replaced by Metrics in the
controller-runtime manager options in version 0.16.0
as part of
e59161ee8f
Updating the same here.

Signed-off-by: karthik-us <ksubrahm@redhat.com>
2023-08-29 13:39:45 +00:00
rakshith-r
f1e9d8084a cephfs: set Pool parameter to empty for Snapshot-backed volumes
Set VolumeOptions.Pool parameter to empty for Snapshot-backed volumes.
This Pool parameter is optional and  only used as 'pool-layout' parameter
during subvolume and subvolume clone create request in cephcsi
and not used for Snapshot-backed volume at all.
It is not saved anywhere for use in subsequent operations after create too.
Therefore, We can set it to empty and not error out.

Signed-off-by: rakshith-r <rar@redhat.com>
2023-08-17 16:13:48 +00:00
riya-singhal31
d432421a88 cephfs: add CSI-Addons support to the cephfs
this commit adds CSI-Addons support to the
cephfs provisioner

Signed-off-by: riya-singhal31 <rsinghal@redhat.com>
2023-07-14 13:24:00 +00:00
Rakshith R
98fdadfde7 rbd: do not execute rbd sparsify when volume is in use
This commit makes sure sparsify() is not run when rbd
image is in use.
Running rbd sparsify with workload doing io and too
frequently is not desirable.
When a image is in use fstrim is run and sparsify will
be run only when image is not mapped.

Signed-off-by: Rakshith R <rar@redhat.com>
2023-07-11 13:48:36 +00:00
Sachin Prabhu
254699cb1a nfs: add support for clients in the StorageClass
The clients parameter in the storage class is used to limit access to
the export to the set of hostnames, networks or ip addresses specified.

Signed-off-by: Sachin Prabhu <sprabhu@redhat.com>
2023-07-06 06:24:11 +00:00
Niels de Vos
f60a358007 rbd: do not try to run resizefs on an encrypted BlockMode volume
When a volume has AccessType=Block and is encrypted with LUKS, a resize
of the filesystem on the (decrypted) block-device is attempted. This
should not be done, as the application that requested the Block volume
is the only authoritive reader/writer of the data.

In particular VirtualMachines that use RBD volumes as a disk, usually
have a partition table on the disk, instead of only a single filesystem.
The `resizefs` command will not be able to resize the filesystem on the
block-device, as it is a partition table.

When `resizefs` fails during NodeStageVolume, the volume is unstaged and
an error is returned.

Resizing an encrypted block-device requires `cryptsetup resize` so that
the LUKS header on the RBD-image is updated with the correct size. But
there is no need to call `resizefs` in this case.

Fixes: 
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2023-07-03 14:54:39 +00:00
riya-singhal31
9ee128123f cleanup: correct the comment in csi-addons/rbd
Signed-off-by: riya-singhal31 <rsinghal@redhat.com>
2023-06-27 14:07:34 +00:00
Rakshith R
1ca6f004a5 rbd: (lastSyncInfo)handle last sync duration being empty case
This commit modifies code to handle last sync duration being
empty & 0,returning nil & 0 on encountering it respectively.
Earlier both case return 0. Test case is added too.

Signed-off-by: Rakshith R <rar@redhat.com>
2023-06-23 11:23:17 +00:00
Yati Padia
2e2e9044c0 rbd: get lastsyncbytes and lastsycduration for volrep
This commit get more information from the description
like lastsyncbytes and lastsyncduration and send them
as a response of getvolumereplicationinfo request.

Signed-off-by: Yati Padia <ypadia@redhat.com>
2023-06-23 07:38:26 +00:00
riya-singhal31
dbdb9086d8 rbd: migration of replication controller server
this commit migrates the replication controller server
from internal/rbd and adds it to csi-addons.

Signed-off-by: riya-singhal31 <rsinghal@redhat.com>
2023-06-23 06:00:40 +00:00
riya-singhal31
cdaa9264eb rbd: migration of replication service to csi-addon
this commit removes grpc import from replication.go
and replaced it with usual errors and passed gRPC
responses in csi-addons

Signed-off-by: riya-singhal31 <rsinghal@redhat.com>
2023-06-22 11:50:54 +00:00
Rakshith R
32b11be88c cephfs: add mountOptions during NodeStage call
Signed-off-by: Rakshith R <rar@redhat.com>
2023-06-19 09:51:40 +00:00
riya-singhal31
b5e68c810e rbd: add unit test for ParseEncryptionOpts
Signed-off-by: riya-singhal31 <rsinghal@redhat.com>
2023-06-06 22:01:26 +00:00
riya-singhal31
347b4d2885 rbd: remove context where its not being used
Signed-off-by: riya-singhal31 <rsinghal@redhat.com>
2023-06-06 22:01:26 +00:00
riya-singhal31
38f5e860e2 rbd: add check for EncryptionTypeNone
this commit adds the validation for encryption
value as false, and sets the type as none

Signed-off-by: riya-singhal31 <rsinghal@redhat.com>
2023-06-06 22:01:26 +00:00
riya-singhal31
92d9785166 cleanup: ErrWaitTimeout is deprecated in k8s 1.27
replaced ErrWaitTimeout with Interrupted

Signed-off-by: riya-singhal31 <rsinghal@redhat.com>
2023-06-06 12:21:43 +00:00
Niels de Vos
b1a4590967 rebase: update sigs.k8s.io/controller-runtime to current version
There is no release for sigs.k8s.io/controller-runtime that supports
Kubernetes v1.27. The main branch has all the required modifications, so
we can use that for the time being.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2023-06-06 12:21:43 +00:00
Niels de Vos
c968f6407d build: address dupword warnings
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2023-06-05 04:49:46 +00:00
Niels de Vos
9e76eceb5e build: address interfacebloat warning for SubVolumeClient
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2023-06-05 04:49:46 +00:00
Niels de Vos
2ab3e40369 build: address interfacer warning
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2023-06-05 04:49:46 +00:00
Niels de Vos
9271375028 build: address gosec warning
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2023-06-05 04:49:46 +00:00
Niels de Vos
b9b8392f71 build: address errorlint warning
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2023-06-05 04:49:46 +00:00
Niels de Vos
dee6be6832 build: address gocritic warnings
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2023-06-05 04:49:46 +00:00
Niels de Vos
9201da0502 build: address gofmt warnings
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2023-06-05 04:49:46 +00:00
Niels de Vos
a6c14c051f build: address golint warnings
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2023-06-05 04:49:46 +00:00
Niels de Vos
e63ebb73c5 build: address nlreturn warnings
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2023-06-05 04:49:46 +00:00
Niels de Vos
53c94efc02 build: address gocritic warnings
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2023-06-05 04:49:46 +00:00
Niels de Vos
81218a69f9 build: address nolintlint errors from new golangci-lint
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2023-06-05 04:49:46 +00:00
Niels de Vos
3d489781a0 rebase: grpc_middleware.WithUnaryServerChain is deprecated
golangci-lint reports that `grpc_middleware.WithUnaryServerChain` is
deprecated and `google.golang.org/grpc.ChainUnaryInterceptor` should be
used instead.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2023-05-10 09:28:26 +00:00
Niels de Vos
8265abc2c9 nfs: add support for secTypes parameter in StorageClass
CephNFS can enable different security flavours for exported volumes.
This can be configured in the optional `secTypes` parameter in the
StorageClass.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2023-05-04 12:21:46 +00:00
Liang Zheng
5a079122f4 rbd: can exit early if image-meta.json does not exist
Signed-off-by: Liang Zheng <zhengliang0901@gmail.com>
2023-05-02 20:36:24 +00:00
Benoît Knecht
1852e977f8 util: Limit cryptsetup PBKDF memory usage
By default, `cryptsetup luksFormat` uses Argon2i as Password-Based Key
Derivation Function (PBKDF), which not only has a CPU cost, but also a memory
cost (to make brute-force attacks harder).

The memory cost is based on the available system memory by default, which in
the context of Ceph CSI can be a problem for two reasons:

1. Pods can have a memory limit (much lower that the memory available on the
   node, usually) which isn't taken into account by `cryptsetup`, so it can get
   OOM-killed when formating a new volume;
2. The amount of memory that was used during `cryptsetup luksFormat` will then
   be needed for `cryptsetup luksOpen`, so if the volume was formated on a node
   with a lot of memory, but then needs to be opened on a different node with
   less memory, `cryptsetup` will get OOM-killed.

This commit sets the PBKDF memory limit to a fixed value to ensure consistent
memory usage regardless of the specifications of the nodes where the volume
happens to be formatted in the first place.

The limit is set to a relatively low value (32 MiB) so that the `csi-rbdplugin`
container in the `nodeplugin` pod doesn't require an extravagantly high memory
limit in order to format/open volumes (particularly with operations happening
in parallel), while at the same time not being so low as to render it
completely pointless.

Signed-off-by: Benoît Knecht <bknecht@protonmail.ch>
2023-04-27 10:43:45 +00:00
riya-singhal31
304194a0c0 cleanup: migration of volrep to csi-addons
This commit moves the volrep logic from internal/rbd to
internal/csi-addons/rbd.

Signed-off-by: riya-singhal31 <rsinghal@redhat.com>
2023-04-21 13:05:20 +00:00
Niels de Vos
37c8f07ed5 rbd: do not run mkfs on a BlockMode volume
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2023-03-08 16:26:39 +00:00
Niels de Vos
a4678200e5 rbd: allow setting mkfsOptions in the StorageClass
Add `mkfsOptions` to the StorageClass and pass them to the `mkfs`
command while creating the filesystem on the RBD device.

Fixes: 
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2023-03-08 16:26:39 +00:00
Niels de Vos
13cdb08e61 rbd: cleanup passing mkfs arguments for NodeStageVolume
Storing the default `mkfs` arguments in a map with key per filesystem
type makes this a little more modular. It prepares th code for fetching
the `mkfs` arguments from the VolumeContext.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2023-03-08 16:26:39 +00:00
riya-singhal31
b28b5e6c84 cephfs: use shallow volumes for the ROX accessMode
this commit makes shallow volume as default feature for ROX volumes.

Signed-off-by: riya-singhal31 <rsinghal@redhat.com>
2023-02-21 20:09:13 +00:00
Rakshith R
95682522ee rbd: add capability to automatically enable read affinity
This commit makes use of crush location labels from node
labels to supply `crush_location` and `read_from_replica=localize`
options during rbd map cmd. Using these options, ceph
will be able to redirect reads to the closest OSD,
improving performance.

Signed-off-by: Rakshith R <rar@redhat.com>
2023-02-14 08:29:46 +00:00
Madhu Rajanna
3967e4dae9 cleanup: fix static checks
fix SA1019 static check to replace
io/utils with os package and sets
with generic sets

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2023-02-03 08:55:43 +00:00
Madhu Rajanna
e9e33fb851 cleanup: fix static checks
fix SA1019 static check to replace
io/utils with os package

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2023-02-02 14:53:59 +00:00
Madhu Rajanna
e54a97ba85 rbd: discover if StagingTargetPath in NodeExpandVolume
The StagingTargetPath is an optional entry in
NodeExpandVolumeRequest, We cannot expect it to be
set always and at the same time cephcsi depended
on the StaingTargetPath to retrieve some metadata
information.

This commit will check all the mount ref and identifies
the stagingTargetPath by checking the image-meta.json
file exists and this is a costly operation as we need to
loop through all the mounts and check image-meta.json
in each mount but this is happens only if the
StaingTargetPath is not set in the NodeExpandVolumeRequest

fixes 

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2023-01-31 08:20:36 +00:00
Madhu Rajanna
d5278bd6c5 rbd: set disableInUseChecks on rbd volume
set disableInUseChecks on rbd volume struct
as it will be used later to check whether
the rbd image is allowed to mount on multiple
nodes.

fixes: 

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2023-01-11 16:24:07 +00:00
Madhu Rajanna
f7796081d3 cephfs: skip expand for BackingSnapshot volume
We should not call ExpandVolume for the BackingSnapshot
subvolume as there wont be any real subvolume created for
it and even if we call it the ExpandVolume will fail
fail as there is no real subvolume exists.

This commits fixes by adjusting the `if` check to ensure
that ExpandVolume will only be called either the
VolumeRequest is to create from a snapshot or volume
and BackingSnapshot is not true.

sample code here https://go.dev/play/p/PI2tNii5tTg

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-12-21 12:29:06 +00:00
Marcel Lauhoff
0bf8646340 cephfs: nolint:gocyclo NewVolumeOptions, NewVolumeOptionsFromVolID
Signed-off-by: Marcel Lauhoff <marcel.lauhoff@suse.com>
2022-11-23 12:21:02 +00:00
Marcel Lauhoff
4788d279a5 cephfs: fscrypt encryption support
Add Ceph FS fscrypt support, similar to the RBD/ext4 fscrypt
integration. Supports encrypted PVCs, snapshots and clones.

Requires kernel and Ceph MDS support that is currently not in any
stable release.

Signed-off-by: Marcel Lauhoff <marcel.lauhoff@suse.com>
2022-11-23 12:21:02 +00:00
Humble Chirammal
71c4ae542c rebase: remove protobuf dependency locking
this commit remove the protobuf dependency locking in the module
description.

Also, ptypes.TimestampProto is deprecated and this commit
make use of the timestamppb.New() for the construction.

ParseTime() function has been removed and callers adjusted to the
same.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-11-15 00:10:46 +00:00
Madhu Rajanna
d12400aa9c rbd: unset metadata if setmetadata is false
We need to unset the metadata on the clone
and restore PVC if the parent PVC was created
when setmetadata was set to true and it was
set to false when restore and clone pvc was
created.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-11-14 14:41:36 +00:00
Rakshith R
eb21d75ef7 rbd: ignore stdErr for ceph osd blocklist when there is no error
`ceph osd blocklist range add/rm <ip>` cmd is outputting
"blocklisting cidr:10.1.114.75:0/32 until 202..." messages
incorrectly into stdErr. This commit ignores stdErr when err
is nil.

Signed-off-by: Rakshith R <rar@redhat.com>
2022-11-12 04:20:14 +00:00
Humble Chirammal
d70b594946 rbd: remove false error check in getDeviceSize
this removed err condition will be always false as error
is always nil.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-11-09 15:35:45 +00:00
Rakshith R
8650538b78 rbd: setup encryption if rbdVol exits during CreateVol
This commit adds code to setup encryption on a rbdVol
being repaired in a followup CreateVolume request.
This is fixes a bug wherein encryption metadata may not
have been set in previous request due to container restart.

Fixes: 

Signed-off-by: Rakshith R <rar@redhat.com>
2022-11-07 12:49:18 +00:00
Madhu Rajanna
07e9dede2c rbd: check volume details from original volumeID
Checking volume details for the existing volumeID
first. if details like OMAP, RBD Image, Pool doesnot
exists try to use clusterIDMapping to look for the
correct informations.

fixes: 

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-11-04 06:32:05 +00:00
Madhu Rajanna
3e1f60244e rbd: check for empty lastSyncTime
Sometime the json unmarshal might
get success and return empty time
stamp. add a check to make sure the
time is not zero always.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-11-03 08:10:19 +00:00
Madhu Rajanna
8f25edc888 rbd: return error if last sync time not present
As per the csiaddon spec last sync time is
required parameter in the GetVolumeReplicationInfo
if we are failed to parse the description, return
not found error message instead of nil
which is empty response

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-11-03 08:10:19 +00:00
Madhu Rajanna
07aa9dea5c rbd: update namespace name in rados object
If a PV is reattached to a new PVC in a different
namespace we need to update the namespace name
in the rados object.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-10-28 15:50:01 +00:00
Madhu Rajanna
019628c8c2 rbd: update namespace name in metadata
If a PV is reattached to a new PVC in a different
namespace we need to update the namespace name
in the rbd image metadata.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-10-28 15:50:01 +00:00
Madhu Rajanna
848e3ee557 rbd: return abnormal in NodeGetVolumeStats
When we do stat on the targetpath, if there is
any error we can check is it due to corruption.
If yes, cephcsi can return abnormal in the
NodeGetVolumeStats so that consumer (CO/admin)
and detect and take further action.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-10-26 09:40:22 +00:00
Madhu Rajanna
44d4546480 cephfs: return abnormal in NodeGetVolumeStats
When we do stat on the targetpath, if there is
any error we can check is it due to corruption.
If yes, cephcsi can return abnormal in the
NodeGetVolumeStats so that consumer (CO/admin)
and detect and take further action.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-10-26 09:40:22 +00:00
Madhu Rajanna
f12fa3ee56 rbd: return GRPC error from GRPC method
GRPC methods should only return GRPC errors
if any error occurs.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-10-19 08:00:42 +00:00
Madhu Rajanna
302fead713 cephfs: delete subvolume if SetAllMetadata fails
To avoid subvolume leaks if the SetAllMetadata
operations fails delete the subvolume.
If any operation fails after creating the subvolume
we will remove the omap as the omap gets
removed we will need to remove the subvolume to
avoid stale resources.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-10-18 15:10:18 +00:00
Marcel Lauhoff
5a55419025 cephfs: Add placeholder journal fscrypt support
Signed-off-by: Marcel Lauhoff <marcel.lauhoff@suse.com>
2022-10-17 17:33:52 +00:00
Marcel Lauhoff
dc7ba684e3 rbd: Use EncryptionTypeNone
Signed-off-by: Marcel Lauhoff <marcel.lauhoff@suse.com>
2022-10-17 17:33:52 +00:00
Marcel Lauhoff
2abfafdf3f util: Add EncryptionTypeNone and unit tests
Add type none to distinguish disabled encryption (positive result)
from invalid configuration (negative result).

Signed-off-by: Marcel Lauhoff <marcel.lauhoff@suse.com>
2022-10-17 17:33:52 +00:00
Marcel Lauhoff
da76d8ddae kms: Add GetSecret() to KMIP KMS
Signed-off-by: Marcel Lauhoff <marcel.lauhoff@suse.com>
2022-10-17 17:33:52 +00:00
Marcel Lauhoff
1f1504479c rbd: Add context to fscrypt errors
Signed-off-by: Marcel Lauhoff <marcel.lauhoff@suse.com>
2022-10-17 17:33:52 +00:00
Marcel Lauhoff
3e3af4da18 rbd: support file encrypted snapshots
Support fscrypt on RBD snapshots

Signed-off-by: Marcel Lauhoff <marcel.lauhoff@suse.com>
2022-10-17 17:33:52 +00:00
Marcel Lauhoff
82d92aab4a rbd: Add volume journal encryption support
Add fscrypt support to the journal to support operations like
snapshotting.

Signed-off-by: Marcel Lauhoff <marcel.lauhoff@suse.com>
2022-10-17 17:33:52 +00:00
Marcel Lauhoff
a7ea12eb8e rbd: Handle encryption type default at a more meaningful place
Different places have different meaningful fallback. When parsing
from user we should default to block, when parsing stored config we
should default to invalid and handle that as an error.

Signed-off-by: Marcel Lauhoff <marcel.lauhoff@suse.com>
2022-10-17 17:33:52 +00:00
Marcel Lauhoff
1fa842277a rbd: fscrypt file encryption support
Integrate basic fscrypt functionality into RBD initialization. To
activate file encryption instead of block introduce the new
'encryptionType' storage class key.

Signed-off-by: Marcel Lauhoff <marcel.lauhoff@suse.com>
2022-10-17 17:33:52 +00:00
Marcel Lauhoff
f1f50e0218 fscrypt: fix metadata directory permissions
Call Mount.Setup with SingleUserWritable constant instead of 0o755,
which is silently ignored and causes the /.fscrypt/{policy,protector}/
directories to have mode 000.

Signed-off-by: Marcel Lauhoff <marcel.lauhoff@suse.com>
2022-10-17 17:33:52 +00:00
Marcel Lauhoff
4e38bdac10 fscrypt: fsync encrypted dir after setting policy [workaround]
Revert once our google/fscrypt dependency is upgraded to a version
that includes https://github.com/google/fscrypt/pull/359 gets accepted

Signed-off-by: Marcel Lauhoff <marcel.lauhoff@suse.com>
2022-10-17 17:33:52 +00:00
Marcel Lauhoff
33c33a8b49 fscrypt: Use constant protector name
Use constant protector name 'ceph-csi' instead of constant prefix
concatenated with the volume ID. When cloning volumes the ID changes
and fscrypt protected directories become inunlockable due to the
protector name change

Signed-off-by: Marcel Lauhoff <marcel.lauhoff@suse.com>
2022-10-17 17:33:52 +00:00
Marcel Lauhoff
97cb1b6672 fscrypt: Update mount info before create context
NewContextFrom{Mountpoint,Path} functions use cached
`/proc/self/mountinfo` to find mounted file systems by device ID.
Since we run fscrypt as a library in a long-lived process the cached
information is likely to be stale. Stale entries may map device IDs to
mount points of already destroyed RBDs and fail context creation.
Updating the cache beforehand prevents this.

Signed-off-by: Marcel Lauhoff <marcel.lauhoff@suse.com>
2022-10-17 17:33:52 +00:00
Marcel Lauhoff
a52314356e fscrypt: Determine best supported fscrypt policy on node init
Currently fscrypt supports policies version 1 and 2. 2 is the best
choice and was the only choice prior to this commit. This adds support
for kernels < 5.4, by selecting policy version 1 there.

Signed-off-by: Marcel Lauhoff <marcel.lauhoff@suse.com>
2022-10-17 17:33:52 +00:00
Marcel Lauhoff
dd0e1988c0 fscrypt: Fetch passphrase when keyFn is invoked not created
Fetch password when keyFn is invoked, not when it is created. This
allows creation of the keyFn before actually creating the passphrase.

Signed-off-by: Marcel Lauhoff <marcel.lauhoff@suse.com>
2022-10-17 17:33:52 +00:00
Marcel Lauhoff
a6a4282493 fscrypt: Unlock: Fetch keys early
Fetch keys from KMS before doing anything else. This will catch KMS
errors before setting up any fscrypt metadata.

Signed-off-by: Marcel Lauhoff <marcel.lauhoff@suse.com>
2022-10-17 17:33:52 +00:00
Marcel Lauhoff
cfea8d7562 fscrypt: fscrypt integration
Integrate google/fscrypt into Ceph CSI KMS and encryption setup. Adds
dependencies to google/fscrypt and pkg/xattr. Be as generic as
possible to support integration with both RBD and Ceph FS.

Add the following public functions:

InitializeNode: per-node initialization steps. Must be called
before Unlock at least once.

Unlock: All steps necessary to unlock an encrypted directory including
setting it up initially.

IsDirectoryUnlocked: Test if directory is really encrypted

Signed-off-by: Marcel Lauhoff <marcel.lauhoff@suse.com>
2022-10-17 17:33:52 +00:00
Marcel Lauhoff
2cf8ecc6c7 journal: Store encryptionType in Config struct
Add encryptionType next to kmsID to support both block and file
encryption.

Signed-off-by: Marcel Lauhoff <marcel.lauhoff@suse.com>
2022-10-17 17:33:52 +00:00
Marcel Lauhoff
ce9fbb3474 rbd: Rename encryption to blockEncryption prep for fscrypt
In preparation of fscrypt support for RBD filesystems, rename block
encryption related function to include the word 'block'. Add struct
fields and IsFileEncrypted.

Signed-off-by: Marcel Lauhoff <marcel.lauhoff@suse.com>
2022-10-17 17:33:52 +00:00
Marcel Lauhoff
624905d60d kms: Add basic GetSecret() test
Add rudimentary test to ensure that we can get a valid passphrase from
the GetSecret() feature

Signed-off-by: Marcel Lauhoff <marcel.lauhoff@suse.com>
2022-10-17 17:33:52 +00:00
Marcel Lauhoff
5df45f1c1b kms: testing: add KMS test dummy registry
Add registry similar to the providers one. This allows testers to
add and use GetKMSTestDummy() to create stripped down provider
instances suitable for use in unit tests.

Signed-off-by: Marcel Lauhoff <marcel.lauhoff@suse.com>
2022-10-17 17:33:52 +00:00
Marcel Lauhoff
cb02a9beb9 kms: Add GetSecret() to metadata KMS
Add GetSecret() to allow direct access to passphrases without KDF and
wrapping by a DEKStore.

This will be used by fscrypt, which has its own KDF and wrapping. It
will allow users to take a k8s secret, for example, and use that
directly as a password in fscrypt.

Signed-off-by: Marcel Lauhoff <marcel.lauhoff@suse.com>
2022-10-17 17:33:52 +00:00
Marcel Lauhoff
0599089de0 util: Add util to fetch encryption type from vol options
Fetch encryption type from vol options. Make fallback type
configurable to support RBD (default block) and Ceph FS (default file)

Signed-off-by: Marcel Lauhoff <marcel.lauhoff@suse.com>
2022-10-17 17:33:52 +00:00
Marcel Lauhoff
fe4821435e util: Make encryption passphrase size a parameter
fscrypt support requires keys longer than 20 bytes. As a preparation,
make the new passphrase length configurable, but default to 20 bytes.

Signed-off-by: Marcel Lauhoff <marcel.lauhoff@suse.com>
2022-10-17 17:33:52 +00:00
Madhu Rajanna
69eb6e40dc rbd: return GRPC error message
The error message return from the GRPC
should be of GRPC error messages only
not the normal go errors. This commits
returns GRPC error if setAllMetadata
fails.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-10-17 15:17:29 +00:00
Madhu Rajanna
01d4a614c3 rbd: delete volume if setallmetadata fails
If any operations fails after the volume creation
we will cleanup the omap objects, but it is missing
if setAllMetadata fails. This commits adds the code
to cleanup the rbd image if metadata operation fails.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-10-17 15:17:29 +00:00
Madhu Rajanna
b40e8894f8 cephfs: use errors.As instead of errors.Is
As we need to compare the error type instead
of the error value we need to use errors.As
to check the API is implemented or not.

fixes: 

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-10-17 09:11:45 +00:00
Niels de Vos
b7703faf37 util: make inode metrics optional in FilesystemNodeGetVolumeStats()
CephFS does not have a concept of "free inodes", inodes get allocated
on-demand in the filesystem.

This confuses alerting managers that expect a (high) number of free
inodes, and warnings get produced if the number of free inodes is not
high enough. This causes alerts to always get reported for CephFS.

To prevent the false-positive alerts from happening, the
NodeGetVolumeStats procedure for CephFS (and CephNFS) will not contain
inodes in the reply anymore.

See-also: https://bugzilla.redhat.com/2128263
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-10-13 19:02:47 +00:00
Madhu Rajanna
71e5b3f922 rbd: remove dummy image workaround
To address the problem that snapshot
schedules are triggered for volumes
that are promoted, a dummy image was
disabled/enabled for replication.
This was done as a workaround, because the
promote operation was not triggering
the schedules for the image being promoted.

The bugs related to the same have been fixed in
RBD mirroring functionality and hence the
workaround  can be removed from the code base.

ceph tracker https://tracker.ceph.com/issues/53914

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-10-10 08:22:10 +00:00
Yati Padia
36b061d426 rbd: get description from remote status
This commit gets the description from remote status
instead of local status.
Local status doesn't have ',' due to which we get
array index out of range panic.

Fixes: 

Signed-off-by: Yati Padia <ypadia@redhat.com>
Co-authored-by: shyam Ranganathan <srangana@redhat.com>
2022-09-14 12:06:01 +00:00
yati1998
b19705f260 rbd: implements getVolumeReplicationInfo
This commit implements getVolumeReplicationInfo
to get the last sync time and update it in volume
replication CR.

Signed-off-by: yati1998 <ypadia@redhat.com>
2022-09-13 14:17:10 +00:00
Rakshith R
a57859dfa4 rbd: use blocklist range cmd, fallback if it fails
This commit adds blocklist range cmd feature,
while fallbacks to old blocklist one ip at a
time if the cmd is invalid(not available).

Signed-off-by: Rakshith R <rar@redhat.com>
2022-09-13 13:10:32 +00:00
Prashanth Dintyala
2a6487cbf5 rbd: create token and use it for vault SA everytime possible
use TokenRequest API by default for vault SA even with K8s versions < 1.24

Signed-off-by: Prashanth Dintyala <vdintyala@nvidia.com>
2022-09-09 10:13:32 +00:00
Madhu Rajanna
76064d8e34 cephfs: retry subvolumegroup creation
Incase the  subvolumegroup is deleted
and recreated we need to restart the
cephcsi provisioner pod to clear cache
that cephcsi maintains. With this PR
if cephcsi sees NotFound error duing
subvolume creation it will reset the cache
for that filesystem so that in next RPC
call cephcsi will try to create the
subvolumegroup again

Ref: https://github.com/rook/rook/issues/10623

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-09-07 18:24:30 +00:00
Madhu Rajanna
e56621cd66 cephfs: fix subvolumegroup creation for multiple fs
In a cluster we can have multiple filesystem
for that we need to have a map of
subvolumegroups to check filesystem is created
nor not.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-09-07 18:24:30 +00:00
Madhu Rajanna
71dbc7dbb4 rbd: map only primary image
If the image is mirroring enabled
and primary consider it for mapping,
if the image is mirroring enabled but
not primary yet. return error message
until the image is marked as primary.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-09-06 10:40:12 +00:00
Madhu Rajanna
038462ff43 cephfs: return success if metadata operation not supported
If the ceph cluster is of older version and doesnot
support metadata operation, Instead of failing
the request return the success if metadata
operation is not supported.

fixes 

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-08-29 18:37:53 +00:00
Rakshith R
40134772a7 rbd: modify stripSecret mechanism in logGRPC()
This commit updates csi-addons spec version
and modifies logging to strip replication
request secret using csi.StripSecret, then
with replication.protosanitizer if the former
fails. This is done in order to make sure
we strip csi and replication format of secrets.

Signed-off-by: Rakshith R <rar@redhat.com>
2022-08-29 11:18:15 +00:00
Rakshith R
f47839d73d rbd: improve kmip verifyResponse() error message
This commit uses %q instead %v in error messages
and adds result reason and message in kmip
verifyresponse().

Signed-off-by: Rakshith R <rar@redhat.com>
2022-08-24 07:58:57 +00:00
Rakshith R
eaa0e14cb2 rbd: fix bug in kmip kms Decrypt function
This commit fixes a bug in kmip kms Decrypt
function, where emd.DEK was fed in a Nonce
instead of emd.Nonce by mistake.

Signed-off-by: Rakshith R <rar@redhat.com>
2022-08-24 07:58:57 +00:00
Niels de Vos
b697b9b0d9 cleanup: replace github.com/pborman/uuid with github.com/google/uuid
The github.com/google/uuid package is used by Kubernetes, and it is part
of the vendor/ directory already. Our usage of github.com/pborman/uuid
can be replaced by github.com/google/uuid, so that
github.com/pborman/uuid can be removed as a dependency.

Closes: 
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-08-22 14:34:25 +00:00
Rakshith R
19e4146fab rbd: add replication capability & service to csiaddons server
csi-addons server will advertise replication capability and
replication service will run with csi-addons server too.

Signed-off-by: Rakshith R <rar@redhat.com>
2022-08-18 08:19:20 +00:00