Commit Graph

1059 Commits

Author SHA1 Message Date
Rakshith R
e46a007961 rebase: update github.com/libopenstorage/secrets to latest
With this update, we no longer import github.com/hashicorp/vault
which now is under BSL license.
https://github.com/hashicorp/vault/blob/main/LICENSE

resolves: #4196

Signed-off-by: Rakshith R <rar@redhat.com>
2023-10-17 11:06:24 +00:00
karthik-us
5ff0607360 e2e: To test 500MB PVC creation
Adding e2e test to check for successful PVC creation
of 500MB.

Signed-off-by: karthik-us <ksubrahm@redhat.com>
2023-10-12 17:31:46 +00:00
karthik-us
5a0eeb882e cephfs: Fix cephfs PVC sizing
Issue:
The RoundOffCephFSVolSize() function omits the fractional
part when calculating the size for cephfs volumes, leading
to the created volume capacity to be lesser than the requested
volume capacity.

Fix:
Consider the fractional part during the size calculation so the
rounded off volume size will be greater than or equal to the
requested volume size.

Signed-off-by: karthik-us <ksubrahm@redhat.com>
Fixes: #4179
2023-10-12 17:31:46 +00:00
Rakshith R
d516a1d66d cephfs: safeguard subVolumeGroupCreated map from race condition
Multiple go-routines may simultaneously create the
subVolumeGroupCreated map or  write into it
for a particular group.

This commit safeguards subVolumeGroupCreated map
from concurrent creation/writes while allowing for multiple
readers.

Signed-off-by: Rakshith R <rar@redhat.com>
2023-10-10 21:04:05 +00:00
Rakshith R
82f1323af4 cephfs: safeguard localClusterState struct from race conditions
Multiple go-routines may simultaneously check for a clusterID's
presence in clusterAdditionalInfo and create an entry if it is
absent. This set of operation needs to be serialized.

Therefore, this commit safeguards clusterAdditionalInfo map
from concurrent writes with a mutex to prevent the above problem.

Signed-off-by: Rakshith R <rar@redhat.com>
2023-10-10 21:04:05 +00:00
Praveen M
0ed7a64191 rbd: update snap RbdImageName in createSnapshot
This PR updates the snapshot RbdImageName in
`createSnapshot` method. This resolves the
incorrect statement logged during snapshot creation.

Signed-off-by: Praveen M <m.praveen@ibm.com>
2023-10-03 11:45:03 +00:00
Praveen M
e504987984 rbd: update snap RbdImageName
This commit updates the snapshot RbdImageName with the clone
RbdImageName before snapshot creation. This will fix the
incorrect log statement.

Signed-off-by: Praveen M <m.praveen@ibm.com>
2023-09-28 11:51:13 +00:00
HF
5411a69839 rbd: fixed all potential crashing when decoding volume ID failed
Signed-off-by: HF <crazytaxii666@gmail.com>
2023-09-06 13:46:22 +00:00
HF
80ad5b6b8f rbd: fixed csi-rbdplugin crashes when decoding volume ID failed
Signed-off-by: HF <crazytaxii666@gmail.com>
2023-09-05 12:08:53 +00:00
Madhu Rajanna
683821c407 rbd: discard not found error from GetMetadata
During ResyncVolume call, discard not found
error from GetMetadata API. If the image gets
resynced the local image creation time will be
lost, if the key is not present in the image
metadata then we can assume that the image is
already resynced.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2023-09-04 05:55:52 +00:00
Madhu Rajanna
771470d975 cephfs: add support for RWX clone from ROX
Add support to create RWX clone from the
ROX clone, in ceph no subvolume clone is
created when ROX clone is created from a
snapshot just a internal ref counter is
added. This PR allows creating a RWX clone
from a ROX clone which allows users to create
RW copy of PVC where cephcsi will identify
the snapshot created for the ROX volume and
creates a subvolume from the CephFS snapshot.

updates: #3603

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2023-08-31 11:51:18 +00:00
Madhu Rajanna
5d034b1124 cephfs: block creation of ROX clone from ROX volume
As there is no usecase currently, blocking the
creation of ROX clone from the ROX volume.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2023-08-31 11:51:18 +00:00
Madhu Rajanna
e013cfed15 rbd: fix resync issue
During the Demote volume store
the image creation timestamp.

During Resync do below operation

* Check image creation timestamp
stored during Demote operation
and current creation timestamp during Resync
and check both are equal and its for
force resync then issue resync
* If the image on both sides is
not in unknown state, check
last_snapshot_timestamp on the
local mirror description, if its present
send volumeReady as false or else return
error message.

If both the images are in up+unknown the
send volumeReady as true.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2023-08-30 09:07:46 +00:00
Madhu Rajanna
4016876c9d cephfs: fix make go-lint error
Not sure why but go-lint is failing
with below error and this fix is required
to make it pass

```
directive `//nolint:staticcheck // See comment above.`
is unused for linter "staticcheck" (nolintlint)
```

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2023-08-30 09:07:46 +00:00
karthik-us
855d09c1cd rebase: replace MetricsBindAddress with Metrics
MetricsBindAddress is replaced by Metrics in the
controller-runtime manager options in version 0.16.0
as part of
e59161ee8f
Updating the same here.

Signed-off-by: karthik-us <ksubrahm@redhat.com>
2023-08-29 13:39:45 +00:00
rakshith-r
f1e9d8084a cephfs: set Pool parameter to empty for Snapshot-backed volumes
Set VolumeOptions.Pool parameter to empty for Snapshot-backed volumes.
This Pool parameter is optional and  only used as 'pool-layout' parameter
during subvolume and subvolume clone create request in cephcsi
and not used for Snapshot-backed volume at all.
It is not saved anywhere for use in subsequent operations after create too.
Therefore, We can set it to empty and not error out.

Signed-off-by: rakshith-r <rar@redhat.com>
2023-08-17 16:13:48 +00:00
riya-singhal31
d432421a88 cephfs: add CSI-Addons support to the cephfs
this commit adds CSI-Addons support to the
cephfs provisioner

Signed-off-by: riya-singhal31 <rsinghal@redhat.com>
2023-07-14 13:24:00 +00:00
Rakshith R
98fdadfde7 rbd: do not execute rbd sparsify when volume is in use
This commit makes sure sparsify() is not run when rbd
image is in use.
Running rbd sparsify with workload doing io and too
frequently is not desirable.
When a image is in use fstrim is run and sparsify will
be run only when image is not mapped.

Signed-off-by: Rakshith R <rar@redhat.com>
2023-07-11 13:48:36 +00:00
Sachin Prabhu
254699cb1a nfs: add support for clients in the StorageClass
The clients parameter in the storage class is used to limit access to
the export to the set of hostnames, networks or ip addresses specified.

Signed-off-by: Sachin Prabhu <sprabhu@redhat.com>
2023-07-06 06:24:11 +00:00
Niels de Vos
f60a358007 rbd: do not try to run resizefs on an encrypted BlockMode volume
When a volume has AccessType=Block and is encrypted with LUKS, a resize
of the filesystem on the (decrypted) block-device is attempted. This
should not be done, as the application that requested the Block volume
is the only authoritive reader/writer of the data.

In particular VirtualMachines that use RBD volumes as a disk, usually
have a partition table on the disk, instead of only a single filesystem.
The `resizefs` command will not be able to resize the filesystem on the
block-device, as it is a partition table.

When `resizefs` fails during NodeStageVolume, the volume is unstaged and
an error is returned.

Resizing an encrypted block-device requires `cryptsetup resize` so that
the LUKS header on the RBD-image is updated with the correct size. But
there is no need to call `resizefs` in this case.

Fixes: #3945
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2023-07-03 14:54:39 +00:00
riya-singhal31
9ee128123f cleanup: correct the comment in csi-addons/rbd
Signed-off-by: riya-singhal31 <rsinghal@redhat.com>
2023-06-27 14:07:34 +00:00
Rakshith R
1ca6f004a5 rbd: (lastSyncInfo)handle last sync duration being empty case
This commit modifies code to handle last sync duration being
empty & 0,returning nil & 0 on encountering it respectively.
Earlier both case return 0. Test case is added too.

Signed-off-by: Rakshith R <rar@redhat.com>
2023-06-23 11:23:17 +00:00
Yati Padia
2e2e9044c0 rbd: get lastsyncbytes and lastsycduration for volrep
This commit get more information from the description
like lastsyncbytes and lastsyncduration and send them
as a response of getvolumereplicationinfo request.

Signed-off-by: Yati Padia <ypadia@redhat.com>
2023-06-23 07:38:26 +00:00
riya-singhal31
dbdb9086d8 rbd: migration of replication controller server
this commit migrates the replication controller server
from internal/rbd and adds it to csi-addons.

Signed-off-by: riya-singhal31 <rsinghal@redhat.com>
2023-06-23 06:00:40 +00:00
riya-singhal31
cdaa9264eb rbd: migration of replication service to csi-addon
this commit removes grpc import from replication.go
and replaced it with usual errors and passed gRPC
responses in csi-addons

Signed-off-by: riya-singhal31 <rsinghal@redhat.com>
2023-06-22 11:50:54 +00:00
Rakshith R
32b11be88c cephfs: add mountOptions during NodeStage call
Signed-off-by: Rakshith R <rar@redhat.com>
2023-06-19 09:51:40 +00:00
riya-singhal31
b5e68c810e rbd: add unit test for ParseEncryptionOpts
Signed-off-by: riya-singhal31 <rsinghal@redhat.com>
2023-06-06 22:01:26 +00:00
riya-singhal31
347b4d2885 rbd: remove context where its not being used
Signed-off-by: riya-singhal31 <rsinghal@redhat.com>
2023-06-06 22:01:26 +00:00
riya-singhal31
38f5e860e2 rbd: add check for EncryptionTypeNone
this commit adds the validation for encryption
value as false, and sets the type as none

Signed-off-by: riya-singhal31 <rsinghal@redhat.com>
2023-06-06 22:01:26 +00:00
riya-singhal31
92d9785166 cleanup: ErrWaitTimeout is deprecated in k8s 1.27
replaced ErrWaitTimeout with Interrupted

Signed-off-by: riya-singhal31 <rsinghal@redhat.com>
2023-06-06 12:21:43 +00:00
Niels de Vos
b1a4590967 rebase: update sigs.k8s.io/controller-runtime to current version
There is no release for sigs.k8s.io/controller-runtime that supports
Kubernetes v1.27. The main branch has all the required modifications, so
we can use that for the time being.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2023-06-06 12:21:43 +00:00
Niels de Vos
c968f6407d build: address dupword warnings
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2023-06-05 04:49:46 +00:00
Niels de Vos
9e76eceb5e build: address interfacebloat warning for SubVolumeClient
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2023-06-05 04:49:46 +00:00
Niels de Vos
2ab3e40369 build: address interfacer warning
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2023-06-05 04:49:46 +00:00
Niels de Vos
9271375028 build: address gosec warning
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2023-06-05 04:49:46 +00:00
Niels de Vos
b9b8392f71 build: address errorlint warning
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2023-06-05 04:49:46 +00:00
Niels de Vos
dee6be6832 build: address gocritic warnings
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2023-06-05 04:49:46 +00:00
Niels de Vos
9201da0502 build: address gofmt warnings
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2023-06-05 04:49:46 +00:00
Niels de Vos
a6c14c051f build: address golint warnings
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2023-06-05 04:49:46 +00:00
Niels de Vos
e63ebb73c5 build: address nlreturn warnings
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2023-06-05 04:49:46 +00:00
Niels de Vos
53c94efc02 build: address gocritic warnings
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2023-06-05 04:49:46 +00:00
Niels de Vos
81218a69f9 build: address nolintlint errors from new golangci-lint
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2023-06-05 04:49:46 +00:00
Niels de Vos
3d489781a0 rebase: grpc_middleware.WithUnaryServerChain is deprecated
golangci-lint reports that `grpc_middleware.WithUnaryServerChain` is
deprecated and `google.golang.org/grpc.ChainUnaryInterceptor` should be
used instead.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2023-05-10 09:28:26 +00:00
Niels de Vos
8265abc2c9 nfs: add support for secTypes parameter in StorageClass
CephNFS can enable different security flavours for exported volumes.
This can be configured in the optional `secTypes` parameter in the
StorageClass.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2023-05-04 12:21:46 +00:00
Liang Zheng
5a079122f4 rbd: can exit early if image-meta.json does not exist
Signed-off-by: Liang Zheng <zhengliang0901@gmail.com>
2023-05-02 20:36:24 +00:00
Benoît Knecht
1852e977f8 util: Limit cryptsetup PBKDF memory usage
By default, `cryptsetup luksFormat` uses Argon2i as Password-Based Key
Derivation Function (PBKDF), which not only has a CPU cost, but also a memory
cost (to make brute-force attacks harder).

The memory cost is based on the available system memory by default, which in
the context of Ceph CSI can be a problem for two reasons:

1. Pods can have a memory limit (much lower that the memory available on the
   node, usually) which isn't taken into account by `cryptsetup`, so it can get
   OOM-killed when formating a new volume;
2. The amount of memory that was used during `cryptsetup luksFormat` will then
   be needed for `cryptsetup luksOpen`, so if the volume was formated on a node
   with a lot of memory, but then needs to be opened on a different node with
   less memory, `cryptsetup` will get OOM-killed.

This commit sets the PBKDF memory limit to a fixed value to ensure consistent
memory usage regardless of the specifications of the nodes where the volume
happens to be formatted in the first place.

The limit is set to a relatively low value (32 MiB) so that the `csi-rbdplugin`
container in the `nodeplugin` pod doesn't require an extravagantly high memory
limit in order to format/open volumes (particularly with operations happening
in parallel), while at the same time not being so low as to render it
completely pointless.

Signed-off-by: Benoît Knecht <bknecht@protonmail.ch>
2023-04-27 10:43:45 +00:00
riya-singhal31
304194a0c0 cleanup: migration of volrep to csi-addons
This commit moves the volrep logic from internal/rbd to
internal/csi-addons/rbd.

Signed-off-by: riya-singhal31 <rsinghal@redhat.com>
2023-04-21 13:05:20 +00:00
Niels de Vos
37c8f07ed5 rbd: do not run mkfs on a BlockMode volume
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2023-03-08 16:26:39 +00:00
Niels de Vos
a4678200e5 rbd: allow setting mkfsOptions in the StorageClass
Add `mkfsOptions` to the StorageClass and pass them to the `mkfs`
command while creating the filesystem on the RBD device.

Fixes: #374
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2023-03-08 16:26:39 +00:00
Niels de Vos
13cdb08e61 rbd: cleanup passing mkfs arguments for NodeStageVolume
Storing the default `mkfs` arguments in a map with key per filesystem
type makes this a little more modular. It prepares th code for fetching
the `mkfs` arguments from the VolumeContext.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2023-03-08 16:26:39 +00:00