we have added clusterID mapping to identify the volumes
in case of a failover in Disaster recovery in #1946.
with #2314 we are moving to a configuration in
configmap for clusterID and poolID mapping.
and with #2314 we have all the required information
to identify the image mappings.
This commit removes the workaround implementation done
in #1946.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
Instead of installing the amd64 on all the
platforms, install architecture specific go
version in test dockerfile.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
Instead of installing the amd64 on all the
platforms, install architecture specific go
version for devel dockerfile
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
currently getting find command not found
and xargs command not found when we run
the dockerfile.test. installing findutils
to fix it.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
As we are updating the cephcsi main repo
to golang 1.17, updating the action package
to golang 1.17
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
as we added kubernetes 1.23 for mergify, removing
older version i.e 1.20 as we will run tests
only on latest 3 kubernetes releases.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
as we are running tests on kubernetes 1.23 by
default, adding mergify condition to check test
passed on kubernetes 1.23 also.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
The rbd package contains several functions that can be used by
CSI-Addons Service implmentations. Unfortunately it is not possible to
do this, as the rbd-driver needs to import the csi-addons/rbd package to
provide the CSI-Addons server. This causes a circular import when
services use the rbd package:
- rbd/driver.go import csi-addons/rbd
- csi-addons/rbd import rbd (including the driver)
By moving rbd/driver.go into its own package, the circular import can be
prevented.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
HexStringToInteger() used to return a uint64, but everywhere else uint
is used. Having HexStringToInteger() return a uint as well makes it a
little easier to use when setting it with SetGlobalInt().
Signed-off-by: Niels de Vos <ndevos@redhat.com>
When the rbd-driver starts, it initializes some global (yuck!) variables
in the rbd package. Because the rbd-driver is moved out into its own
package, these variables can not easily be set anymore.
Introcude SetGlobalInt(), SetGlobalBool() and InitJournals() so that the
rbd-driver can configure the rbd package.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
The rbd-driver calls rbd.runVolumeHealer() which is not available
outside the rbd package. By moving the rbd-driver into its own package,
RunVolumeHealer() needs to be exported.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
NodeServer.mounter is internal to the NodeServer type, but it needs to
be initialized by the rbd-driver. The rbd-driver is moved to its own
package, so .Mounter needs to be available from there in order to set
it.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
genVolFromVolID() is used by the CSI Controller service to create an
rbdVolume object from a CSI volume_id. This function is useful for
CSI-Addons Services as well, so rename it to GenVolFromVolID().
Signed-off-by: Niels de Vos <ndevos@redhat.com>
k8s.io/utils/mount has moved to k8s.io/mount-utils, and Ceph-CSI uses
that already in most locations. Only internal/util/util.go still imports
the old path.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
adding e2e testcase to validate the workflow
of pvc creation and attaching to pod works for
new image features like fast-diff,obj-map,exclusive-lock
and layering.
fixes: #2695
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
The dummy image will be created with 1Mib size.
during the snapshot transfer operation the 1Mib
will be transferred even if the dummy image doesnot
contains any data. adding the new image features
`fast-diff,layering,obj-map,exclusive-lock`on the
dummy image will ensure that only the diff is
transferred to the remote cluster.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
we added a workaround for rbd scheduling by creating
a dummy image in #2656. with the fix we are creating
a dummy image of the size of the first actual rbd
image which is sent in EnableVolumeReplication request
if the actual rbd image size is 1TiB we are creating
a dummy image of 1TiB which is not good. even though
its a thin provisioned rbd images this is causing
issue for the transfer of the snapshot during
the mirroring operation.
This commit recreates the rbd image with 1MiB size
which is the smaller supported size in rbd.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
Currently, we are skipping the generic ephemeral
testing if the kubernetes version is less than
1.21 because of this one the who test suite is
getting skipped and e2e is marked as success
in 2 minutes. This commit runs the ephemeral
tests if the kube=>1.21+. If we do this, for
the lower version we can run other tests.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
rbd mirroring CLI calls are async and it doesn't wait
for the operation to be completed. ex:- `rbd mirror image enable`
it will enable the mirroring on the image but it doesn't
ensure that the image is mirroring enabled and healthy
primary. The same goes for the promote volume also.
This commits adds a check-in PromoteVolume to make sure
the image in a healthy state i.e `up+stopped`.
note:- not considering any intermediate states to make
sure the image is completely healthy before responding
success to the RPC call.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
Journal-based RADOS block device mirroring ensures point-in-time
consistent replicas of all changes to an image, including reads and
writes, block device resizing, snapshots, clones, and flattening.
Journaling-based mirroring records all modifications to an image in the
order in which they occur. This ensures that a crash-consistent mirror
of an image is available.
Mirroring when configured in journal mode, mirroring will
utilize the RBD journaling image feature to replicate the image
contents. If the RBD journaling image feature is not yet enabled on the
image, it will be automatically enabled.
Fixes: #2018
Co-authored-by: Madhu Rajanna <madhupr007@gmail.com>
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
Depending on the way Ceph-CSI is deployed, the capabilities will be
configured for the GetCapabilities procedure. The other procedures are
more straight-forward.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
After adding the new CSI-Addons Server, golang-ci complains that
driver.Run() is too complex. By moving the profiling checks and starting
of the go-routines in their own function, golang-ci is happy again.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Add a new CSI-Addons Server and empty Identity Service for the RBD
plugin. The implementation of the Identity Service procedure calls will
be done in other PRs.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
When generating csiconfiguration from values the config.json key gets merged with cluster-mapping.json
as the config.json toYaml element supresses a newline.
This fixes the situation where configuration is generated as shown;
```
data:
config.json: |-
[{"clusterID":"....","monitors":["..."]}]cluster-mapping.json: |-
[]
```
Signed-off-by: Toby Jackson <toby@warmfusion.co.uk>
Github's list statuses returns list of all status, possibly
containing dupicates and previously failed statuses.
Use "UpdatedAt" timestamp to only get the latest status
for each test.
Signed-off-by: Rakshith R <rar@redhat.com>
Version field for helm Chart.yaml needs to have SemVer 2
compatible value, therefore use "<MAJOR-VERSION>-canary"
on "devel" branch.
Refer: https://helm.sh/docs/topics/charts/#the-chartyaml-file
Signed-off-by: Rakshith R <rar@redhat.com>
The e2e sometimes fail getting objects like PVCs from the Kubernetes API
server, and log the following error:
Error getting pvc "rbd-6940" in namespace "rbd-694": rpc error: code = Unknown desc = OK: HTTP status code 200; transport: missing content-type field
By checking the error message, and initiating a retry on this failure,
CI jobs should fail less regulary.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
It seems that the matching condition on the modified files with
-files~=^(!?actions/)
validates to 'true' when that is not intended. The example in the
Mergify documentation does not seem to be correct :-/
This reverts commit 411bf33a3d.
See-also: https://docs.mergify.com/examples/#merging-based-on-modified-files
Signed-off-by: Niels de Vos <ndevos@redhat.com>
PRs that contain `mergify` or `Mergify` in the subject should both match
the rule to add labels. The regular expressions can include `(?i)` to
make the matching case-insensitive.
See-also: https://docs.mergify.com/configuration/#regular-expressions
Signed-off-by: Niels de Vos <ndevos@redhat.com>
The matching checks the list of modified files, and if it does not
contain anything outside the `actions/` directory, the labels are
applied.
Actions are not tested with e2e, so changes can be labelled to skip
these time and resource intensive tests.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Add tests for RWX and ROX accessModes for Block and FileSystem Mode
PVCs.
Fixes: #2262
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
