all the error check scenarios of genVolFromVolID() and unreserving
omap entries based on the error made deleteVolume method complex,
this patch create a new function which handle the error check and
unrerving omap entries accordingly and finally return the response
to deletevolume/caller.
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
When NewK8sClient() detects and error, it used to call FatalLogMsg()
which causes a panic. There are additional features that can be used on
Kubernetes clusters, but these are not a requirement for most
functionalities of the driver.
Instead of causing a panic, returning an error should suffice. This
allows using the driver on non-Kubernetes clusters again.
Fixes: #2452
Signed-off-by: Niels de Vos <ndevos@redhat.com>
There is no use in having Mergify posting messages to Dependabot and
asking for resolving of merge conflicts. Dependabot will try to do that
automatically, and posts a message when some other action is needed.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
google.golang.org/grpc was listed in go.mod:replace and updating the
dependency was not possible. There should not be a reason to prevent
updating, commit 5aca04d540 added it but without explanation.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
The commit messages that Dependabot (@app/dependabot) creates are not
always accepted by the commitlint check. The configuration for
dependabot does not give a lot of options to customize the message, so
instead of adjusting the message to pass commitlint, just skip the check
(Mergify does not require commitlint status either).
See-also: #2460
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Dependabot can not update the dependencies for k8s.io/kubernetes
correctly. Helping the bot out with this additional commit.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
CephFS csi driver explictly set the size of the cloned volume
to the size of parent volume as cephfs mgr was lacking this
functionality previously. However it has been addressed in cephfs
so we dont need explicit size setting.
Ref#https://tracker.ceph.com/issues/46163
Supported Ceph releases:
Ceph versions equal or above - v16.0.0, v15.2.9, v14.2.12
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
As we have deprecated earlier versions than v3.3.0, it is not required
to keep the upgrade docs for the same. The upgrade doc for v3.2.0 to
v3.3.0 has been kept intact.
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
This commit adds fetchMappedClusterIDAndMons() which returns
monitors and clusterID info after checking cluster mapping info.
This is required for regenerating omap entries in mirrored cluster
with different clusterID.
Signed-off-by: Rakshith R <rar@redhat.com>
This commit moves getMappedID() from rbd to util
package since it is not rbd specific and exports
it from there.
Signed-off-by: Rakshith R <rar@redhat.com>
These dependencies are pulled in by k8s.io/kubernetes with version
v0.0.0. It is therefore required to use 'replace' in go.mod to select a
compatible version of the additional k8s.io packages.
Dependabot does not seem to update packages listed in 'replace', only
under 'require'. That means, the version updates done by Dependabot do
not have any effect, as the contents is replaced with a different
version anyway. Ignoring these packages prevents the creation of
non-functional PRs.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Dependabot creates commit messages that commitlint fails to parse
correctly. The messages contain simple references to which dependencies
are updated, and do not need full validation like manual written
messages do.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
CSI version v3.3.0 and above are the supported versions now and this
patch also adjust the mergify rules according to that, ie the backport
rules are removed for all unsupported versions here.
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
A new "internal/kms" package is introduced, it holds the API that can be
consumed by the RBD components.
The KMS providers are currently in the same package as the API. With
later follow-up changes the providers will be placed in their own
sub-package.
Because of the name of the package "kms", the types, functions and
structs inside the package should not be prefixed with KMS anymore:
internal/kms/kms.go:213:6: type name will be used as kms.KMSInitializerArgs by other packages, and that stutters; consider calling this InitializerArgs (golint)
Updates: #852
Signed-off-by: Niels de Vos <ndevos@redhat.com>
By placing the NewK8sClient() function in its own package, the KMS API
can be split from the "internal/util" package. Some of the KMS providers
use the NewK8sClient() function, and this causes circular dependencies
between "internal/utils" -> "internal/kms" -> "internal/utils", which
are not alowed in Go.
Updates: #852
Signed-off-by: Niels de Vos <ndevos@redhat.com>
There is a problem accessing the code.cloudfoundry.org web service iver
TLS. It seems to redirect to GitHub, so use the package from there:
running: go mod verify
go: github.com/libopenstorage/secrets@v0.0.0-20210709082113-dde442ea20ec requires
github.com/hashicorp/vault@v1.4.2 requires
github.com/hashicorp/vault-plugin-auth-cf@v0.5.4 requires
github.com/cloudfoundry-community/go-cfclient@v0.0.0-20190201205600-f136f9222381 requires
code.cloudfoundry.org/gofileutils@v0.0.0-20170111115228-4d0c80011a0f: unrecognized import path "code.cloudfoundry.org/gofileutils": https fetch: Get "https://code.cloudfoundry.org/gofileutils?go-get=1": x509: certificate signed by unknown authority
Signed-off-by: Niels de Vos <ndevos@redhat.com>
The `make containerized-test TARGET=mod-check` jobs fail because the web
service at gomodules.xyz is not behaving correctly at the moment:
running: go mod verify
go: sigs.k8s.io/controller-runtime@v0.9.2 requires
gomodules.xyz/jsonpatch/v2@v2.2.0: unrecognized import path "gomodules.xyz/jsonpatch/v2": reading https://gomodules.xyz/jsonpatch/v2?go-get=1: 404 Not Found
Signed-off-by: Niels de Vos <ndevos@redhat.com>
e2elog.Logf("waiting for kubectl (%s -f $q args %s) to finish", action, args)
changed to
e2elog.Logf("waiting for kubectl (%s -f args %s) to finish", action, args)
Signed-off-by: Rakshith R <rar@redhat.com>
if the volumeattachment has been fetched but marked for deletion
the nbd healer dont want to process further on this pv. This patch
adds a check for pv is marked for deletion and if so, make the
healer skip processing the same
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
As discussed in https://github.com/ceph/ceph-csi/issues/2438
we are marking ceph csi release support to N.(x-1) release versions.
N = latest major release
x = latest minor release
This address the release version support matrix based on the
same.
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
Moving the log functions into its own internal/util/log package makes it
possible to split out the humongous internal/util packages in further
smaller pieces. This reduces the inter-dependencies between utility
functions and components, preventing circular dependencies which are not
allowed in Go.
Updates: #852
Signed-off-by: Niels de Vos <ndevos@redhat.com>
added github action to check for the
stale issues and PRs. the action will
get scheduled everydata at 21:00 UTC.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
This commit removes the stale bot
configration as stale bot repo is not actively
maintained anymore.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
Unit-testing often fails due to a race condition while writing the
clusterMappingConfigFile from multiple go-routines at the same time.
Failures from `make containerized-test` look like this:
=== CONT TestGetClusterMappingInfo/site2-storage_cluster-id_mapping
cluster_mapping_test.go:153: GetClusterMappingInfo() = <nil>, expected data &[{map[site1-storage:site2-storage] [map[1:3]] [map[11:5]]} {map[site3-storage:site2-storage] [map[8:3]] [map[10:5]]}]
=== CONT TestGetClusterMappingInfo/site3-storage_cluster-id_mapping
cluster_mapping_test.go:153: GetClusterMappingInfo() = <nil>, expected data &[{map[site3-storage:site2-storage] [map[8:3]] [map[10:5]]}]
--- FAIL: TestGetClusterMappingInfo (0.01s)
--- PASS: TestGetClusterMappingInfo/mapping_file_not_found (0.00s)
--- PASS: TestGetClusterMappingInfo/mapping_file_found_with_empty_data (0.00s)
--- PASS: TestGetClusterMappingInfo/cluster-id_mapping_not_found (0.00s)
--- FAIL: TestGetClusterMappingInfo/site2-storage_cluster-id_mapping (0.00s)
--- FAIL: TestGetClusterMappingInfo/site3-storage_cluster-id_mapping (0.00s)
--- PASS: TestGetClusterMappingInfo/site1-storage_cluster-id_mapping (0.00s)
By splitting the public GetClusterMappingInfo() function into an
internal getClusterMappingInfo() that takes a filename, unit-testing can
use different files for each go-routine, and testing becomes more
predictable.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
updated mergify rules to consider the teams
approval to merge a PR.
more details at #2367fixes#2367
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
We need
https://www.mail-archive.com/linux-block@vger.kernel.org/msg38060.html
inorder to use `--io-timeout=0`. This patch is part of kernel 5.4
Since minikube doesn't have a v5.4 kernel yet, lets use io-timeout value
conditionally based on kernel version at our e2e.
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
Currently, we get the kernel version where the e2e (client) executable runs,
not the kernel version that is used by the csi-rbdplugin pod.
Add a function that run `uname -r` command from the specified container and
returns the kernel version.
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
Suggested-by: Niels de Vos <ndevos@redhat.com>
With the tests at CI, it kind of looks like that the IO is timing out after
30 seconds (default with rbd-nbd). Since we have tweaked reattach-timeout
to 300 seconds at ceph-csi, we need to explicitly set io-timeout on the
device too, as it doesn't make any sense to keep
io-timeout < reattach-timeout
Hence we set io-timeout for rbd nbd to 0. Specifying io-timeout 0 tells
the nbd driver to not abort the request and instead see if it can be
restarted on another socket.
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
Suggested-by: Ilya Dryomov <idryomov@redhat.com>
- Update the meta stash with logDir details
- Use the same to remove logfile on unstage/unmap to be space efficient
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
- One logfile per device/volume
- Add ability to customize the logdir, default: /var/log/ceph
Note: if user customizes the hostpath to something else other than default
/var/log/ceph, then it is his responsibility to update the `cephLogDir`
in storageclass to reflect the same with daemon:
```
cephLogDir: "/var/log/mynewpath"
```
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
Problem:
--------
1. rbd-nbd by default logs to /var/log/ceph/ceph-client.admin.log,
Unfortunately, container doesn't have /var/log/ceph directory hence
rbd-nbd is not logging now.
2. Rbd-nbd logs are not persistent across nodeplugin restarts.
Solution:
--------
Provide a host path so that log directory is made available, and the
logs persist on the hostnode across container restarts.
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
Ceph’s logging levels operate on a scale of 1 to 20, where 1 is terse
and 20 is verbose.
Format:
debug-{subsystem} = {log-level}
Setting `rbd` loglevel to 20 at our e2e tests.
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
There are TODO and FIXME comments in the Ceph-CSI source code that need
addressing at one point. Adding this TODO badge and link to tickgit to
the main README makes it obvious that some cleanup is needed.
This might invite new contributors to address reported TODOs.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
pylint started to report errors like the following:
troubleshooting/tools/tracevol.py:97:10: R1732: Consider using 'with' for resource-allocating operations (consider-using-with)
There probably has been an update of Pylint in the test-container that
is more strict than previous versions.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
