Commit Graph

3472 Commits

Author SHA1 Message Date
yati1998
b19705f260 rbd: implements getVolumeReplicationInfo
This commit implements getVolumeReplicationInfo
to get the last sync time and update it in volume
replication CR.

Signed-off-by: yati1998 <ypadia@redhat.com>
2022-09-13 14:17:10 +00:00
Rakshith R
a57859dfa4 rbd: use blocklist range cmd, fallback if it fails
This commit adds blocklist range cmd feature,
while fallbacks to old blocklist one ip at a
time if the cmd is invalid(not available).

Signed-off-by: Rakshith R <rar@redhat.com>
2022-09-13 13:10:32 +00:00
dependabot[bot]
3d04065d00 rebase: bump k8s.io/klog/v2 from 2.70.1 to 2.80.1
Bumps [k8s.io/klog/v2](https://github.com/kubernetes/klog) from 2.70.1 to 2.80.1.
- [Release notes](https://github.com/kubernetes/klog/releases)
- [Changelog](https://github.com/kubernetes/klog/blob/main/RELEASE.md)
- [Commits](https://github.com/kubernetes/klog/compare/v2.70.1...v2.80.1)

---
updated-dependencies:
- dependency-name: k8s.io/klog/v2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-09-13 07:56:49 +00:00
Prashanth Dintyala
2a6487cbf5 rbd: create token and use it for vault SA everytime possible
use TokenRequest API by default for vault SA even with K8s versions < 1.24

Signed-off-by: Prashanth Dintyala <vdintyala@nvidia.com>
2022-09-09 10:13:32 +00:00
Madhu Rajanna
76064d8e34 cephfs: retry subvolumegroup creation
Incase the  subvolumegroup is deleted
and recreated we need to restart the
cephcsi provisioner pod to clear cache
that cephcsi maintains. With this PR
if cephcsi sees NotFound error duing
subvolume creation it will reset the cache
for that filesystem so that in next RPC
call cephcsi will try to create the
subvolumegroup again

Ref: https://github.com/rook/rook/issues/10623

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-09-07 18:24:30 +00:00
Madhu Rajanna
e56621cd66 cephfs: fix subvolumegroup creation for multiple fs
In a cluster we can have multiple filesystem
for that we need to have a map of
subvolumegroups to check filesystem is created
nor not.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-09-07 18:24:30 +00:00
Madhu Rajanna
9d46478794 ci: use resync to sync helm charts
When a file on source is deleted same
need to be deleted on the destination,
with rsync we can achieve it.

fixes: #3329

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-09-06 11:22:51 +00:00
Madhu Rajanna
71dbc7dbb4 rbd: map only primary image
If the image is mirroring enabled
and primary consider it for mapping,
if the image is mirroring enabled but
not primary yet. return error message
until the image is marked as primary.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-09-06 10:40:12 +00:00
dependabot[bot]
9a54f05e66 rebase: bump google.golang.org/grpc from 1.48.0 to 1.49.0
Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.48.0 to 1.49.0.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.48.0...v1.49.0)

---
updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-09-05 16:32:37 +00:00
Humble Chirammal
586a9cc8ee rbd: change default FsGroupPolicy to "File" for RBD CSI driver
This commit change the default fsgroup policy for csi driver object
to "File" type which is the better/correct setting for the CSI volumes.
We have been using default value which is "ReadWriteOnceWithFSType".
with this change backward compatibility should be preserved.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-09-05 07:06:37 +00:00
Madhu Rajanna
038462ff43 cephfs: return success if metadata operation not supported
If the ceph cluster is of older version and doesnot
support metadata operation, Instead of failing
the request return the success if metadata
operation is not supported.

fixes #3347

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-08-29 18:37:53 +00:00
Rakshith R
40134772a7 rbd: modify stripSecret mechanism in logGRPC()
This commit updates csi-addons spec version
and modifies logging to strip replication
request secret using csi.StripSecret, then
with replication.protosanitizer if the former
fails. This is done in order to make sure
we strip csi and replication format of secrets.

Signed-off-by: Rakshith R <rar@redhat.com>
2022-08-29 11:18:15 +00:00
Madhu Rajanna
93f6923084 e2e: use ginkgo.timeout for e2e
ginkgo.timeout is introduced in
ginkgo timeout for overall test suite

https://onsi.github.io/ginkgo/
\MIGRATING_TO_V2#timeout-behavior

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-08-25 16:36:35 +00:00
Madhu Rajanna
ade8b91516 rebase: update gingko to v2 version
Ginkgo v1 is deprecated and was replaced
with the v2.

Ref: https://onsi.github.io/ginkgo/
MIGRATING_TO_V2#upgrading-to-ginkgo-20

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-08-25 16:36:35 +00:00
Madhu Rajanna
e3bf375035 rebase: update kubernetes to v1.25.0
update kubernetes to latest v1.25.0
release.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-08-25 16:36:35 +00:00
Rakshith R
f47839d73d rbd: improve kmip verifyResponse() error message
This commit uses %q instead %v in error messages
and adds result reason and message in kmip
verifyresponse().

Signed-off-by: Rakshith R <rar@redhat.com>
2022-08-24 07:58:57 +00:00
Rakshith R
eaa0e14cb2 rbd: fix bug in kmip kms Decrypt function
This commit fixes a bug in kmip kms Decrypt
function, where emd.DEK was fed in a Nonce
instead of emd.Nonce by mistake.

Signed-off-by: Rakshith R <rar@redhat.com>
2022-08-24 07:58:57 +00:00
Madhu Rajanna
96a3aabe5a deploy: remove psp from cephcsi
as PSP is deprecated in kubernetes 1.21
and will be removed in kubernetes 1.25
removing the existing PSP related templates
from the repo and updated the required documents.

fixes #1988

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-08-23 07:53:46 +00:00
dependabot[bot]
3d05ef0585 rebase: bump github.com/aws/aws-sdk-go from 1.44.67 to 1.44.82
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.67 to 1.44.82.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Changelog](https://github.com/aws/aws-sdk-go/blob/main/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.44.67...v1.44.82)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-08-23 04:46:49 +00:00
Niels de Vos
b697b9b0d9 cleanup: replace github.com/pborman/uuid with github.com/google/uuid
The github.com/google/uuid package is used by Kubernetes, and it is part
of the vendor/ directory already. Our usage of github.com/pborman/uuid
can be replaced by github.com/google/uuid, so that
github.com/pborman/uuid can be removed as a dependency.

Closes: #3315
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-08-22 14:34:25 +00:00
dependabot[bot]
aad99ebcb7 rebase: bump github.com/onsi/gomega from 1.19.0 to 1.20.0
Bumps [github.com/onsi/gomega](https://github.com/onsi/gomega) from 1.19.0 to 1.20.0.
- [Release notes](https://github.com/onsi/gomega/releases)
- [Changelog](https://github.com/onsi/gomega/blob/master/CHANGELOG.md)
- [Commits](https://github.com/onsi/gomega/compare/v1.19.0...v1.20.0)

---
updated-dependencies:
- dependency-name: github.com/onsi/gomega
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-08-22 07:06:12 +00:00
Rakshith R
6e100dde2d ci: skip mutli-arch-build for doc and mergify prs
Signed-off-by: Rakshith R <rar@redhat.com>
2022-08-19 15:42:16 +02:00
Rakshith R
45f8232256 doc: add nfs upgrade guide from 3.6 to 3.7
This commit adds nfs upgrade guide from 3.6
to 3.7.

Closes: #3271

Signed-off-by: Rakshith R <rar@redhat.com>
2022-08-19 04:47:11 +00:00
Humble Chirammal
8d3e63a2b2 build: remove release 3.5 backport rule
considering we have 3.7 release in place, this commit remove
the release 3.5 backporting based on the labels in the repo.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-08-19 08:12:27 +05:30
Humble Chirammal
7e44e557f0 build: add mergify rule for backport PRs in release 3.7 branch
This commit add rule for backporting to release 3.7 branch with
label backport-to-release-v3.7

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-08-19 08:12:27 +05:30
Humble Chirammal
2505c2ad14 rebase: use latest kubernetes version
This commit rebase the kubernetes version to 1.24.4 tree to make
sure we are using latest available bug fix release.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-08-18 21:46:46 +00:00
Humble Chirammal
1e2a777d67 doc: update documentation for release 3.7.0
This commit add upgrade documentation for release 3.7.0
and also update support matrix for v3.7.0.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-08-18 15:58:45 +00:00
Humble Chirammal
a9446c328e doc: correct e2e documentation formatting
This commit address formatting and alligment issues in e2e README.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-08-18 12:47:53 +00:00
Humble Chirammal
5cb8f80c10 build: enable ceph_pre_quincy tag
This commit add ceph_pre_quincy tag to the build. The main
reason being the changes or new APIs exposed in go-ceph version
v0.17.0 is part of this Ceph build tag. Enabling the same for
the reason.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-08-18 11:42:57 +00:00
Humble Chirammal
483181aec2 rebase: use v1.17.0 of go-ceph library
new version of go ceph is available and this commit make use
of the same.
Ref # https://github.com/ceph/go-ceph/releases/tag/v0.17.0

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-08-18 11:42:57 +00:00
Rakshith R
19e4146fab rbd: add replication capability & service to csiaddons server
csi-addons server will advertise replication capability and
replication service will run with csi-addons server too.

Signed-off-by: Rakshith R <rar@redhat.com>
2022-08-18 08:19:20 +00:00
Rakshith R
cb9ec35e3a rebase: update csi-addons/spec to latest
csi-addons/spec is updated to latest to
get replication identity capability.

Signed-off-by: Rakshith R <rar@redhat.com>
2022-08-18 08:19:20 +00:00
Rakshith R
e72ed593be rebase: vendor files required for kmip
Signed-off-by: Rakshith R <rar@redhat.com>
2022-08-18 07:41:42 +00:00
Rakshith R
0c33a33d5c rbd: add kmip encryption type
The Key Management Interoperability Protocol (KMIP)
is an extensible communication protocol
that defines message formats for the manipulation
of cryptographic keys on a key management server.
Ceph-CSI can now be configured to connect to
various KMS using KMIP for encrypting RBD volumes.

https://en.wikipedia.org/wiki/Key_Management_Interoperability_Protocol

Signed-off-by: Rakshith R <rar@redhat.com>
2022-08-18 07:41:42 +00:00
dependabot[bot]
2fc10ded65 rebase: bump github.com/aws/aws-sdk-go-v2/service/sts
Bumps [github.com/aws/aws-sdk-go-v2/service/sts](https://github.com/aws/aws-sdk-go-v2) from 1.16.9 to 1.16.13.
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/v1.16.9...service/ivs/v1.16.13)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/service/sts
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-08-16 09:34:29 +00:00
Madhu Rajanna
842279b811 ci: use ubuntu-latest for stale job
Currently, we use the Ubuntu 18.04 actions runner
for stale job. This runner will be deprecated
and removed in the beginning of Dec.
So should change the runner to use latest ubuntu.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-08-16 07:00:44 +00:00
Madhu Rajanna
dde21543bd cephfs: fix staticcheck comment
getting is unused for linter "staticcheck"
(nolintlint) error message due to wrong
comment format. this the format now with
`//directive // comment`

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-08-10 17:51:26 +00:00
Madhu Rajanna
5c9f63bada ci: disable nosnakecase static check
All the CSI spec imports are having snakecase
we cannot fix it, instead of disabling all imports
skipping the nosnakecase static check.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-08-10 17:51:26 +00:00
Madhu Rajanna
a3751ada24 ci: update dependencies in build.env
This commits updates below items

* Golang version to v1.18.5
* Golangci-lint to v1.47.3
* Helm version to v3.9.2
* Rook version to v1.9.8

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-08-10 17:51:26 +00:00
Rakshith R
54fe4158a0 deploy: update fsGroupPolicy to File for nfs driver object
NFS volume similar to CephFs volumes can support
fsGroupPolicy as File, now Kubernetes Kubernetes may
use fsGroup to change permissions and ownership of the
volume to match user requested fsGroup in the pod's
SecurityPolicy regardless of fstype or access mode.

Signed-off-by: Rakshith R <rar@redhat.com>
2022-08-09 13:36:03 +00:00
Rakshith R
d39d2cffcc cleanup: use index instead of value while iterating
This commit cleans up for loop to use index to access
value instead of copying value into a new variable
while iterating.
```
internal/util/csiconfig.go:103:2: rangeValCopy: each \
iteration copies 136 bytes (consider pointers or indexing) \
(gocritic)
        for _, cluster := range config {
```

Signed-off-by: Rakshith R <rar@redhat.com>
2022-08-09 13:36:03 +00:00
Rakshith R
48d66d6cfd deploy: modify nfs daemonset to use cephcsi nfs nodeserver
This commit makes modification to nfs daemonset to use
nfs nodeserver. `nfs.NetNamespaceFilePath` example is
added.

Signed-off-by: Rakshith R <rar@redhat.com>
2022-08-09 13:36:03 +00:00
Rakshith R
3d3c029471 nfs: add nodeserver within cephcsi
This commit adds nfs nodeserver capable of
mounting nfs volumes, even with pod networking
using NSenter design similar to rbd and cephfs.
NodePublish, NodeUnpublish, NodeGetVolumeStats
and NodeGetCapabilities have been implemented.

The nodeserver implementation has been inspired
from https://github.com/kubernetes-csi/csi-driver-nfs,
which was previously used for mounted cephcsi exported
nfs volumes. The current implementation is also
backward compatible for the previously created
PVCs.

Signed-off-by: Rakshith R <rar@redhat.com>
2022-08-09 13:36:03 +00:00
Shyamsundar Ranganathan
c2280011d1 rbd: Report remote peer readiness if Up and status.Unknown
Current code uses an !A && !B condition incorrectly to
test A:Up and B:status for a remote peer image.

This should be !A || !B as we require both conditions to
be in the specified state (Up: true, and status Unknown).

This is corrected by this commit, and further fixes:
- check and return ready only when a remote site is
found in the status output
- check if all peer sites are ready, if multiple are found
and return ready appropriately

Signed-off-by: Shyamsundar Ranganathan <srangana@redhat.com>
2022-08-09 05:32:15 +00:00
Madhu Rajanna
8d7b6ee59f rbd: consider mirror deamon state for ResyncVolume
During ResyncVolume we check if the image
is in an error state, and we resync.
After resync, the image will move to
either the `Error` or the `Resyncing` state.
And if the image is in the above two
conditions, we will return a successful
response and Ready=false so that the
consumer can wait until the volume is
ready to use. If the image is in any
other state we return an error message
to indicate the syncing is not going on.
The whole resync and image state change
depends on the rbd mirror daemon. If the
mirror daemon is not running, the image
can be in Resyncing or Unknown state.
The Ramen marks the volume replication as
secondary, and once the resync starts, it
will delete the volume replication CR as a
cleanup process.

As we dont have a check for the rbd mirror
daemon, we are returning a resync success
response and Ready=false. Due to this false
response Ramen is assuming the resync started
and deleted the volume replication CR, and
because of this, the cluster goes into a bad
state and needs manual intervention.

fixes #3289

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-08-08 13:26:15 +00:00
Rakshith R
1ea4a1b790 ci: fix invalid mergifyio configuration
Comment out `comment: ` settings, since it
does not have any options set, otherwise
throws the following error.
```
The current Mergify configuration is invalid
required key not provided @ defaults → actions → comment → message
```

Signed-off-by: Rakshith R <rar@redhat.com>
2022-08-08 13:35:30 +05:30
Humble Chirammal
c9773db3f3 ci: remove check for snapshot controller installation and cleanup
At present, the check is performed to validate the version of kube
is v1.17 and this commit remove the same.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-08-05 12:02:45 +00:00
Humble Chirammal
5aabd4e1d2 deploy: remove the snapshot controller installation check
no need to have 1.17 kube version check anymore  before we install
snapshot controller.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-08-05 12:02:45 +00:00
Madhu Rajanna
297b14ed54 ci: update minikube to v1.26.1
update minikube to latest patch release.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-08-04 17:26:56 +00:00
Niels de Vos
83df1eae53 rebase: k8s.io/mount-utils/IsNotMountPoint() is deprecated
IsNotMountPoint() is deprecated and Mounter.IsMountPoint() is
recommended to be used instead.

Reported-by: golangci/staticcheck
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-08-04 09:53:07 +00:00