Commit Graph

574 Commits

Author SHA1 Message Date
Niels de Vos
9b41feac9d csiaddons: initial implementation of CSI-Addons VolumeGroup
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-07-11 11:17:13 +00:00
Niels de Vos
d248a1e200 rbd: add Manager interface for using Volumes and VolumeGroups
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-07-11 11:17:13 +00:00
Niels de Vos
4246b30178 rbd: add the VolumeGroup type
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-07-11 11:17:13 +00:00
Niels de Vos
2803ec1290 rbd: add a ToCSI() function to the Volume interface
A VolumeGroup CSI-Addons object contains a list of CSI Volumes. A
ToCSI() function makes creating such a list much simpler.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-07-11 11:17:13 +00:00
Madhu Rajanna
2dd2ac8e91 csiaddons: register volumegroup controller
Register the volumegroup controller as part
of rbd controller server to serve the volume
group RPC spec.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-07-11 11:17:13 +00:00
Niels de Vos
6e5de23674 rbd: move internal/rbd_types -> internal/rbd/types
The rbd_types package was initially created with references to the rbd
package. And the rbd package references the rbd_types package. Having
rbd/types was not possible due to recursive imports. After cleaning up
the rbd_types package, it can be renamed to rbd/types.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-07-11 11:17:13 +00:00
Niels de Vos
902946ebfa rbd: add Volume interface
The Volume interface will make it easier to work with the rbdImage
struct, as the functions are cleaner defined. This benefits work that is
needed for VolumeGroups and other CSI-Addons procedures.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-07-08 09:23:58 +00:00
Niels de Vos
8d5e1aad4e rbd: rename deleteImage() to Delete()
Renaming the `Delete()` function makes it easier to implement the
upcoming `Volume` interface.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-07-08 09:23:58 +00:00
Niels de Vos
3aece2f38e cleanup: pass context to rbdImage.Destroy()
In the future we'll introduce a more standard interface for objects like
Volumes and Snapshots. It is useful to have the context passed as 1st
argument to all functions of those objects, including their Destroy()
function.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-07-08 09:23:58 +00:00
Rakshith R
d166229d8f rbd: add support for flattenMode option for replication
This commit adds support for flattenMode option
for replication.
If the flattenMode is set to "force" in
volumereplicationclass parameters, cephcsi will
add a task to flatten the image if it has parent.
This enable cephcsi to then mirror such images after
flattening them.
The error message when the image's parent is
in trash or unmirrored is improved as well.

Signed-off-by: Rakshith R <rar@redhat.com>
2024-06-20 14:51:36 +00:00
Rakshith R
ec8017512f rbd: use internal as default error code in getGRPCError()
This commit replaces codes.Unknown with codes.Internal
as the default error code in getGRPCError().

Signed-off-by: Rakshith R <rar@redhat.com>
2024-06-11 15:17:53 +00:00
Mayank Pal
822794c242 ci: Use temporary directory for unit tests
ensure a clean and isolated environment for testing purposes.

Signed-off-by: Mayank Pal <mayankpal9654@gmail.com>

ci: Use temporary directory for unit tests

remove err = os.Mkdir('/etc/ceph-csi-config', 0o600)

Signed-off-by: Mayank Pal <mayankpal9654@gmail.com>

ci: Use temporary directory for unit tests

remove err = os.Mkdir('/etc/ceph-csi-config', 0o600)

Signed-off-by: Mayank Pal <mayankpal9654@gmail.com>

ci: Use temporary directory for unit tests

remove if err

Signed-off-by: Mayank Pal <mayankpal9654@gmail.com>
2024-05-27 05:57:27 +00:00
Mike Perez
c8af2b638a util: Removing JoinError in favor of fmt.Errorf
Signed-off-by: Mike Perez <thingee@gmail.com>
2024-05-22 09:05:42 +00:00
Niels de Vos
e85914fc0d build: address 'intrange' linter warning
golangci-lint warns about this:

    for loop can be changed to use an integer range (Go 1.22+)
    (intrange)

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-04-29 14:36:48 +00:00
Niels de Vos
0e7b06e9d0 build: address 'copyloopvar' linter warning
golangci-lint reports these:

    The copy of the 'for' variable "kmsID" can be deleted (Go 1.22+)
    (copyloopvar)

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-04-29 14:36:48 +00:00
Rakshith R
1bb78fdf43 e2e: validate PVC-PVC clone creation with deleted parent snap
This commit modifies a test case to check creation of
PVC-PVC clone of a restored PVC when parent snapshot
is deleted.

Signed-off-by: Rakshith R <rar@redhat.com>
2024-04-23 12:04:59 +00:00
Rakshith R
c34b31ee05 rbd: add ParentInTrash parameter in rbdImage struct
This commit adds ParentInTrash parameter in rbdImage struct
and makes use of it in getParent() function in order to avoid
error in case the parent is present but in trash.

Signed-off-by: Rakshith R <rar@redhat.com>
2024-04-23 12:04:59 +00:00
Madhu Rajanna
4c2d2caf9f util: add support to configure mirror daemon count
Currently we are assuming that only one
rbd mirror daemon running on the ceph cluster
but that is not true for many cases and it
can be more that one, this PR make this as a
configurable parameter.

fixes: #4312

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-04-22 09:49:59 +00:00
Madhu Rajanna
8c4a38eec6 rbd: address golangci-lint issues
addressing golangci-lint issues in rbd
related code.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2024-04-09 06:55:23 +00:00
Praveen M
c1467242c6 cleanup: use slices package
This commit replaces the user implemented function
`CheckSliceContains()` with `slices.Contains()`
function introduced in Go 1.21.

Signed-off-by: Praveen M <m.praveen@ibm.com>
2024-04-05 12:18:00 +00:00
Praveen M
3538b23794 rbd: remove topologyConstrainedPools parameter
This commit removes the `topologyConstrainedPools` parameter
from PV volumeAttributes as it is not required.

Signed-off-by: Praveen M <m.praveen@ibm.com>
2024-04-05 12:18:00 +00:00
Niels de Vos
5a6556c4d4 cleanup: destroy connections after .Copy() an other one
Everytime a connection is copied with the .Copy() function, it needs to
be destroyed once the object is not needed anymore. This was not done
consistently, a few more locations require the freeing of the connection
resources.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-04-02 11:11:20 +00:00
Niels de Vos
3df396e6f1 rbd: add extra logging while cleaning up snapshots
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-03-28 11:54:28 +00:00
Niels de Vos
ba05c0f5f1 cleanup: reformat generateVolFromSnap() to rbdSnapshot.toVolume()
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-03-28 11:54:28 +00:00
Niels de Vos
a517290ea7 rbd: let parseVolCreateRequest() return a connected rbdVolume
By returning a connected rbdVolume in parseVolCreateRequest(), the
CreateVolume() function can be simplified a little. There is no need to
call the additional Connect() and detect failures with it.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-03-28 11:54:28 +00:00
Niels de Vos
7b2b125b18 rbd: free snapshot resources after allocation
Not all snapshot objects are free'd correctly after they were allocated.
It is possible that some connections to the Ceph cluster were never
closed. This does not need to be a noticeable problem, as connections
are re-used where possible, but it isn't clean either.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-03-28 11:54:28 +00:00
Niels de Vos
18162c71bc cleanup: do not pass an empty snapshot to genSnapFromSnapID()
Just like GenVolFromVolID() the genSnapFromSnapID() function can return
a snapshot. There is no need to allocated an empty snapshot and pass
that to the genSnapFromSnapID() function.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-03-28 11:54:28 +00:00
parth-gr
063319f6e5 rbd: make pool optional in rbd sc if topologyconstraints are present
if rbd storage class is created with topologyconstraintspools
replicated pool was still mandatory, making the pool optional if the
topologyconstraintspools is requested

Closes: https://github.com/ceph/ceph-csi/issues/4380

Signed-off-by: parth-gr <partharora1010@gmail.com>
2024-03-22 13:15:50 +00:00
Niels de Vos
991343d9e5 cleanup: do not pass EncodingVersion to GenerateVolID()
The only encoding version that exists is `1`. There is no need to have
multiple constants for that version across different packages. Because
there is only one version, `GenerateVolID()` does not really require it,
and it can use a default version.

If there is a need in the future to support an other encoding version,
this can be revisited with a cleaner solution.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-03-18 06:23:28 +00:00
muxuelan
4f04748c87 rbd: support nbd on euler or arm
Signed-off-by: muxuelan <muxuelan@cmss.chinamobile.com>
2024-03-15 10:39:50 +00:00
Praveen M
e345b26340 cleanup: refactor functions to accept a context parameter
Signed-off-by: Praveen M <m.praveen@ibm.com>
2024-03-12 13:54:19 +00:00
Niels de Vos
3bf5c0e478 cleanup: simplify rbdGetDeviceList()
The `rbdGetDeviceList()` function uses two very similar types for
converting krbd and NBD device information from JSON. There is no need
to use this distinction, and callers of `rbdGetDeviceList()` should not
need to care about it either.

By introducing a `deviceInfo` interface with Get-functions, the
`rbdGetDeviceList()` function becomes a little simpler, with a clearly
defined API for the returned list.

Signed-off-by: Niels de Vos <ndevos@ibm.com>
2024-01-11 18:00:41 +00:00
Praveen M
50e505619c deploy: added json field tags for csi config map
This commit adds the json field tags for csi config map for
encoding and decoding JSON.

Signed-off-by: Praveen M <m.praveen@ibm.com>
2023-12-21 17:44:46 +00:00
Jan Nemcik
3443546370 rbd: updated node labels fetching logic
node labels are fetched only if controller is running in k8s and is nodeserver

Signed-off-by: Jan Nemcik <jan.nemcik@solargis.com>
2023-12-11 10:59:50 +00:00
Riya Singhal
4b5cdd5316 util: addresed few todo
this commit replaces string comparsion with error code
at few places

Signed-off-by: Riya Singhal <rsinghal@redhat.com>
2023-11-23 00:55:17 +00:00
Praveen M
4d466843b9 cephfs: add read affinity mount option
This commit makes use of crush location labels from node
labels to supply `crush_location` and `read_from_replica=localize`
options during mount. Using these options, cephfs
will be able to redirect reads to the closest OSD,
improving performance.

Signed-off-by: Praveen M <m.praveen@ibm.com>
2023-11-22 13:13:01 +00:00
Praveen M
c4e373c72f deploy: support for read affinity options per cluster
Implemented the capability to include read affinity options
for individual clusters within the ceph-csi-config ConfigMap.
This allows users to configure the crush location for each
cluster separately. The read affinity options specified in
the ConfigMap will supersede those provided via command line arguments.

Signed-off-by: Praveen M <m.praveen@ibm.com>
2023-11-08 21:17:00 +00:00
Madhu Rajanna
304462c7cc cleanup: fix spellcheck errors
fixed spellcheck errors caught in
CI.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2023-11-08 12:32:06 +00:00
Madhu Rajanna
9f753889ed rbd: remove deprecated rbdImageRequiresEncryption
remove support for deprecated
rbdImageRequiresEncryption case.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2023-11-04 08:14:51 +00:00
Madhu Rajanna
3ea540bf0f util: remove deprecated grpc metrics
This commit removes the deprecated
grpc related code from cephcsi.

fixes: #4122

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2023-11-03 10:04:50 +00:00
Praveen M
0ed7a64191 rbd: update snap RbdImageName in createSnapshot
This PR updates the snapshot RbdImageName in
`createSnapshot` method. This resolves the
incorrect statement logged during snapshot creation.

Signed-off-by: Praveen M <m.praveen@ibm.com>
2023-10-03 11:45:03 +00:00
Praveen M
e504987984 rbd: update snap RbdImageName
This commit updates the snapshot RbdImageName with the clone
RbdImageName before snapshot creation. This will fix the
incorrect log statement.

Signed-off-by: Praveen M <m.praveen@ibm.com>
2023-09-28 11:51:13 +00:00
HF
5411a69839 rbd: fixed all potential crashing when decoding volume ID failed
Signed-off-by: HF <crazytaxii666@gmail.com>
2023-09-06 13:46:22 +00:00
HF
80ad5b6b8f rbd: fixed csi-rbdplugin crashes when decoding volume ID failed
Signed-off-by: HF <crazytaxii666@gmail.com>
2023-09-05 12:08:53 +00:00
Madhu Rajanna
e013cfed15 rbd: fix resync issue
During the Demote volume store
the image creation timestamp.

During Resync do below operation

* Check image creation timestamp
stored during Demote operation
and current creation timestamp during Resync
and check both are equal and its for
force resync then issue resync
* If the image on both sides is
not in unknown state, check
last_snapshot_timestamp on the
local mirror description, if its present
send volumeReady as false or else return
error message.

If both the images are in up+unknown the
send volumeReady as true.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2023-08-30 09:07:46 +00:00
Rakshith R
98fdadfde7 rbd: do not execute rbd sparsify when volume is in use
This commit makes sure sparsify() is not run when rbd
image is in use.
Running rbd sparsify with workload doing io and too
frequently is not desirable.
When a image is in use fstrim is run and sparsify will
be run only when image is not mapped.

Signed-off-by: Rakshith R <rar@redhat.com>
2023-07-11 13:48:36 +00:00
Niels de Vos
f60a358007 rbd: do not try to run resizefs on an encrypted BlockMode volume
When a volume has AccessType=Block and is encrypted with LUKS, a resize
of the filesystem on the (decrypted) block-device is attempted. This
should not be done, as the application that requested the Block volume
is the only authoritive reader/writer of the data.

In particular VirtualMachines that use RBD volumes as a disk, usually
have a partition table on the disk, instead of only a single filesystem.
The `resizefs` command will not be able to resize the filesystem on the
block-device, as it is a partition table.

When `resizefs` fails during NodeStageVolume, the volume is unstaged and
an error is returned.

Resizing an encrypted block-device requires `cryptsetup resize` so that
the LUKS header on the RBD-image is updated with the correct size. But
there is no need to call `resizefs` in this case.

Fixes: #3945
Signed-off-by: Niels de Vos <ndevos@ibm.com>
2023-07-03 14:54:39 +00:00
riya-singhal31
dbdb9086d8 rbd: migration of replication controller server
this commit migrates the replication controller server
from internal/rbd and adds it to csi-addons.

Signed-off-by: riya-singhal31 <rsinghal@redhat.com>
2023-06-23 06:00:40 +00:00
riya-singhal31
cdaa9264eb rbd: migration of replication service to csi-addon
this commit removes grpc import from replication.go
and replaced it with usual errors and passed gRPC
responses in csi-addons

Signed-off-by: riya-singhal31 <rsinghal@redhat.com>
2023-06-22 11:50:54 +00:00
riya-singhal31
b5e68c810e rbd: add unit test for ParseEncryptionOpts
Signed-off-by: riya-singhal31 <rsinghal@redhat.com>
2023-06-06 22:01:26 +00:00