Commit Graph

3210 Commits

Author SHA1 Message Date
Madhu Rajanna
4face6a7b3 cephfs: retry subvolumegroup creation
Incase the  subvolumegroup is deleted
and recreated we need to restart the
cephcsi provisioner pod to clear cache
that cephcsi maintains. With this PR
if cephcsi sees NotFound error duing
subvolume creation it will reset the cache
for that filesystem so that in next RPC
call cephcsi will try to create the
subvolumegroup again

Ref: https://github.com/rook/rook/issues/10623

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit 76064d8e34)
2022-09-08 11:37:05 +00:00
Madhu Rajanna
e08143a88b cephfs: fix subvolumegroup creation for multiple fs
In a cluster we can have multiple filesystem
for that we need to have a map of
subvolumegroups to check filesystem is created
nor not.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit e56621cd66)
2022-09-08 11:37:05 +00:00
Madhu Rajanna
468c73d2b6 ci: use resync to sync helm charts
When a file on source is deleted same
need to be deleted on the destination,
with rsync we can achieve it.

fixes: #3329

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit 9d46478794)
2022-09-07 11:30:25 +00:00
Madhu Rajanna
04879bbb33 rbd: map only primary image
If the image is mirroring enabled
and primary consider it for mapping,
if the image is mirroring enabled but
not primary yet. return error message
until the image is marked as primary.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit 71dbc7dbb4)
2022-09-06 14:05:08 +00:00
Humble Chirammal
66512b6a73 rbd: change default FsGroupPolicy to "File" for RBD CSI driver
This commit change the default fsgroup policy for csi driver object
to "File" type which is the better/correct setting for the CSI volumes.
We have been using default value which is "ReadWriteOnceWithFSType".
with this change backward compatibility should be preserved.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
(cherry picked from commit 586a9cc8ee)
2022-09-06 06:17:07 +00:00
Madhu Rajanna
3092b46774 cephfs: return success if metadata operation not supported
If the ceph cluster is of older version and doesnot
support metadata operation, Instead of failing
the request return the success if metadata
operation is not supported.

fixes #3347

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit 038462ff43)
2022-08-30 03:43:33 +00:00
Rakshith R
3a4c7c9d79 rbd: modify stripSecret mechanism in logGRPC()
This commit updates csi-addons spec version
and modifies logging to strip replication
request secret using csi.StripSecret, then
with replication.protosanitizer if the former
fails. This is done in order to make sure
we strip csi and replication format of secrets.

Signed-off-by: Rakshith R <rar@redhat.com>
(cherry picked from commit 40134772a7)
2022-08-29 14:55:24 +00:00
Rakshith R
2f393f24b7 rbd: improve kmip verifyResponse() error message
This commit uses %q instead %v in error messages
and adds result reason and message in kmip
verifyresponse().

Signed-off-by: Rakshith R <rar@redhat.com>
(cherry picked from commit f47839d73d)
2022-08-24 08:32:50 +00:00
Rakshith R
f3675f4f28 rbd: fix bug in kmip kms Decrypt function
This commit fixes a bug in kmip kms Decrypt
function, where emd.DEK was fed in a Nonce
instead of emd.Nonce by mistake.

Signed-off-by: Rakshith R <rar@redhat.com>
(cherry picked from commit eaa0e14cb2)
2022-08-24 08:32:50 +00:00
Humble Chirammal
0dde0e32b1 deploy: revert template changes of 3.7.0 to canary
Helm deployments and other deploy scripts are updated to have
canary tag in the release version

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-08-22 09:05:39 +00:00
Humble Chirammal
34fd27bbd1 deploy: change image versions to v3.7.0 instead of canary
This commit change the required image tag to release 3.7 instead
of canary for v3.7 release

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-08-19 13:25:11 +00:00
Humble Chirammal
aaa3d4e1d0 helm: update image tag for release 3.7 instead of canary
This commit change the image tag for release v3.7 instead of
canary.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-08-19 13:25:11 +00:00
Rakshith R
e4a280f06e doc: add nfs upgrade guide from 3.6 to 3.7
This commit adds nfs upgrade guide from 3.6
to 3.7.

Closes: #3271

Signed-off-by: Rakshith R <rar@redhat.com>
(cherry picked from commit 45f8232256)
2022-08-19 08:25:32 +00:00
Humble Chirammal
ff37d5fae8 rebase: use latest kubernetes version
This commit rebase the kubernetes version to 1.24.4 tree to make
sure we are using latest available bug fix release.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
(cherry picked from commit 2505c2ad14)
2022-08-19 13:08:25 +05:30
Humble Chirammal
efdc6b4572 doc: update documentation for release 3.7.0
This commit add upgrade documentation for release 3.7.0
and also update support matrix for v3.7.0.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
(cherry picked from commit 1e2a777d67)
2022-08-19 03:44:09 +00:00
Humble Chirammal
a9446c328e doc: correct e2e documentation formatting
This commit address formatting and alligment issues in e2e README.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-08-18 12:47:53 +00:00
Humble Chirammal
5cb8f80c10 build: enable ceph_pre_quincy tag
This commit add ceph_pre_quincy tag to the build. The main
reason being the changes or new APIs exposed in go-ceph version
v0.17.0 is part of this Ceph build tag. Enabling the same for
the reason.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-08-18 11:42:57 +00:00
Humble Chirammal
483181aec2 rebase: use v1.17.0 of go-ceph library
new version of go ceph is available and this commit make use
of the same.
Ref # https://github.com/ceph/go-ceph/releases/tag/v0.17.0

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-08-18 11:42:57 +00:00
Rakshith R
19e4146fab rbd: add replication capability & service to csiaddons server
csi-addons server will advertise replication capability and
replication service will run with csi-addons server too.

Signed-off-by: Rakshith R <rar@redhat.com>
2022-08-18 08:19:20 +00:00
Rakshith R
cb9ec35e3a rebase: update csi-addons/spec to latest
csi-addons/spec is updated to latest to
get replication identity capability.

Signed-off-by: Rakshith R <rar@redhat.com>
2022-08-18 08:19:20 +00:00
Rakshith R
e72ed593be rebase: vendor files required for kmip
Signed-off-by: Rakshith R <rar@redhat.com>
2022-08-18 07:41:42 +00:00
Rakshith R
0c33a33d5c rbd: add kmip encryption type
The Key Management Interoperability Protocol (KMIP)
is an extensible communication protocol
that defines message formats for the manipulation
of cryptographic keys on a key management server.
Ceph-CSI can now be configured to connect to
various KMS using KMIP for encrypting RBD volumes.

https://en.wikipedia.org/wiki/Key_Management_Interoperability_Protocol

Signed-off-by: Rakshith R <rar@redhat.com>
2022-08-18 07:41:42 +00:00
dependabot[bot]
2fc10ded65 rebase: bump github.com/aws/aws-sdk-go-v2/service/sts
Bumps [github.com/aws/aws-sdk-go-v2/service/sts](https://github.com/aws/aws-sdk-go-v2) from 1.16.9 to 1.16.13.
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/v1.16.9...service/ivs/v1.16.13)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/service/sts
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-08-16 09:34:29 +00:00
Madhu Rajanna
842279b811 ci: use ubuntu-latest for stale job
Currently, we use the Ubuntu 18.04 actions runner
for stale job. This runner will be deprecated
and removed in the beginning of Dec.
So should change the runner to use latest ubuntu.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-08-16 07:00:44 +00:00
Madhu Rajanna
dde21543bd cephfs: fix staticcheck comment
getting is unused for linter "staticcheck"
(nolintlint) error message due to wrong
comment format. this the format now with
`//directive // comment`

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-08-10 17:51:26 +00:00
Madhu Rajanna
5c9f63bada ci: disable nosnakecase static check
All the CSI spec imports are having snakecase
we cannot fix it, instead of disabling all imports
skipping the nosnakecase static check.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-08-10 17:51:26 +00:00
Madhu Rajanna
a3751ada24 ci: update dependencies in build.env
This commits updates below items

* Golang version to v1.18.5
* Golangci-lint to v1.47.3
* Helm version to v3.9.2
* Rook version to v1.9.8

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-08-10 17:51:26 +00:00
Rakshith R
54fe4158a0 deploy: update fsGroupPolicy to File for nfs driver object
NFS volume similar to CephFs volumes can support
fsGroupPolicy as File, now Kubernetes Kubernetes may
use fsGroup to change permissions and ownership of the
volume to match user requested fsGroup in the pod's
SecurityPolicy regardless of fstype or access mode.

Signed-off-by: Rakshith R <rar@redhat.com>
2022-08-09 13:36:03 +00:00
Rakshith R
d39d2cffcc cleanup: use index instead of value while iterating
This commit cleans up for loop to use index to access
value instead of copying value into a new variable
while iterating.
```
internal/util/csiconfig.go:103:2: rangeValCopy: each \
iteration copies 136 bytes (consider pointers or indexing) \
(gocritic)
        for _, cluster := range config {
```

Signed-off-by: Rakshith R <rar@redhat.com>
2022-08-09 13:36:03 +00:00
Rakshith R
48d66d6cfd deploy: modify nfs daemonset to use cephcsi nfs nodeserver
This commit makes modification to nfs daemonset to use
nfs nodeserver. `nfs.NetNamespaceFilePath` example is
added.

Signed-off-by: Rakshith R <rar@redhat.com>
2022-08-09 13:36:03 +00:00
Rakshith R
3d3c029471 nfs: add nodeserver within cephcsi
This commit adds nfs nodeserver capable of
mounting nfs volumes, even with pod networking
using NSenter design similar to rbd and cephfs.
NodePublish, NodeUnpublish, NodeGetVolumeStats
and NodeGetCapabilities have been implemented.

The nodeserver implementation has been inspired
from https://github.com/kubernetes-csi/csi-driver-nfs,
which was previously used for mounted cephcsi exported
nfs volumes. The current implementation is also
backward compatible for the previously created
PVCs.

Signed-off-by: Rakshith R <rar@redhat.com>
2022-08-09 13:36:03 +00:00
Shyamsundar Ranganathan
c2280011d1 rbd: Report remote peer readiness if Up and status.Unknown
Current code uses an !A && !B condition incorrectly to
test A:Up and B:status for a remote peer image.

This should be !A || !B as we require both conditions to
be in the specified state (Up: true, and status Unknown).

This is corrected by this commit, and further fixes:
- check and return ready only when a remote site is
found in the status output
- check if all peer sites are ready, if multiple are found
and return ready appropriately

Signed-off-by: Shyamsundar Ranganathan <srangana@redhat.com>
2022-08-09 05:32:15 +00:00
Madhu Rajanna
8d7b6ee59f rbd: consider mirror deamon state for ResyncVolume
During ResyncVolume we check if the image
is in an error state, and we resync.
After resync, the image will move to
either the `Error` or the `Resyncing` state.
And if the image is in the above two
conditions, we will return a successful
response and Ready=false so that the
consumer can wait until the volume is
ready to use. If the image is in any
other state we return an error message
to indicate the syncing is not going on.
The whole resync and image state change
depends on the rbd mirror daemon. If the
mirror daemon is not running, the image
can be in Resyncing or Unknown state.
The Ramen marks the volume replication as
secondary, and once the resync starts, it
will delete the volume replication CR as a
cleanup process.

As we dont have a check for the rbd mirror
daemon, we are returning a resync success
response and Ready=false. Due to this false
response Ramen is assuming the resync started
and deleted the volume replication CR, and
because of this, the cluster goes into a bad
state and needs manual intervention.

fixes #3289

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-08-08 13:26:15 +00:00
Rakshith R
1ea4a1b790 ci: fix invalid mergifyio configuration
Comment out `comment: ` settings, since it
does not have any options set, otherwise
throws the following error.
```
The current Mergify configuration is invalid
required key not provided @ defaults → actions → comment → message
```

Signed-off-by: Rakshith R <rar@redhat.com>
2022-08-08 13:35:30 +05:30
Humble Chirammal
c9773db3f3 ci: remove check for snapshot controller installation and cleanup
At present, the check is performed to validate the version of kube
is v1.17 and this commit remove the same.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-08-05 12:02:45 +00:00
Humble Chirammal
5aabd4e1d2 deploy: remove the snapshot controller installation check
no need to have 1.17 kube version check anymore  before we install
snapshot controller.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-08-05 12:02:45 +00:00
Madhu Rajanna
297b14ed54 ci: update minikube to v1.26.1
update minikube to latest patch release.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-08-04 17:26:56 +00:00
Niels de Vos
83df1eae53 rebase: k8s.io/mount-utils/IsNotMountPoint() is deprecated
IsNotMountPoint() is deprecated and Mounter.IsMountPoint() is
recommended to be used instead.

Reported-by: golangci/staticcheck
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-08-04 09:53:07 +00:00
Niels de Vos
10b2277330 util: use k8s.io/mount-utils/NewWithoutSystemd() to prevent logging
NewWithoutSystemd() has been introduced in the k8s.io/mount-utils
package so that systemd is not called while executing functions. This
offers consumers the ability to prevent confusing and scary messages
from getting logged.

See-also: kubernetes/kubernetes#111218
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-08-04 09:53:07 +00:00
Niels de Vos
3a200b6976 rbd: use IsLikelyNotMountPoint() to prevent systemd log messages
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-08-04 09:53:07 +00:00
Niels de Vos
533994daff rebase: update k8s.io/mount-utils to current master
kubernetes/kubernetes#111083 has been merged and synced into
k8s.io/mount-utils. This should remove any systemd log messages while
calling NodeStageVolume and NodeGetVolumeStats.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-08-04 09:53:07 +00:00
Pedro Alvarez
3c3cbc8005 doc: update relative path to ceph-config.yaml file
Signed-off-by: Pedro Alvarez <pedro.alvarez@softiron.com>
2022-08-04 07:16:56 +00:00
Niels de Vos
0a173a8a9e nfs: make DeleteVolume (more) idempotent
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-08-03 19:43:16 +00:00
Niels de Vos
a6cd56ae7e e2e: correct failure logging for NFS
Some of the steps still refer to CephFS, likely missed some replacements
while copy/pasting. The logging is a little confusing when messages
claim something with CephFS failed, but the test is about NFS.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2022-08-03 19:43:16 +00:00
Yati Padia
f0074a3ebf deploy: enable HonorPVReclaimPolicy feature gate
This commit enables the HonorPVReclaimPolicy feature
gate.

fixes: #3230

Signed-off-by: Yati Padia <ypadia@redhat.com>
2022-08-03 19:43:16 +00:00
Humble Chirammal
bc9ad3d9f1 rbd: add dummy attacher implementation
previously, it was a requirement to have attacher sidecar for CSI
drivers and there had an implementation of dummy mode of operation.
However skipAttach implementation has been stabilized and the dummy
mode of operation is going to be removed from the external-attacher.
Considering this driver  work on volumeattachment objects for NBD driver
use cases, we have to implement dummy controllerpublish and unpublish
and thus keep supporting our operations even in absence of dummy mode
of operation in the sidecar.

This commit make a NOOP controller publish and unpublish for RBD driver.

CephFS driver does not require attacher and it has already been made free
from the attachment operations.

    Ref# https://github.com/ceph/ceph-csi/pull/3149
    Ref# https://github.com/kubernetes-csi/external-attacher/issues/226

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-08-03 00:25:49 +00:00
Prasanna Kumar Kalever
b4f44a43d5 doc: Add documentation about --setmetadata option
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2022-08-01 07:15:29 +00:00
Prasanna Kumar Kalever
c0a566b5ed deploy: add setmetadata=true in the templates
setmetadata on the volume by default, otherwise e2e will fail

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2022-08-01 07:15:29 +00:00
Prasanna Kumar Kalever
30244bf11b cephfs: snapshots honor --setmetadata option
`--setmetadata` is false by default, honoring it
will keep the metadata disabled by default

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2022-08-01 07:15:29 +00:00
Prasanna Kumar Kalever
14d6211d6d cephfs: subvolumes honor --setmetadata option
`--setmetadata` is false by default, honoring it
will keep the metadata disabled by default

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2022-08-01 07:15:29 +00:00