Commit Graph

3228 Commits

Author SHA1 Message Date
Madhu Rajanna
d39b61334a cephfs: delete subvolume if SetAllMetadata fails
To avoid subvolume leaks if the SetAllMetadata
operations fails delete the subvolume.
If any operation fails after creating the subvolume
we will remove the omap as the omap gets
removed we will need to remove the subvolume to
avoid stale resources.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit 302fead713)
2022-10-19 07:14:43 +00:00
Niels de Vos
763aa3df03 rebase: ParseAcceptLanguage takes a long time to parse complex tags
A vulnerability was found in golang.org/x/text/language package which
could cause a denial of service. An attacker can craft an
Accept-Language header which ParseAcceptLanguage will take significant
time to parse.
Version v0.3.8 of golang.org/x/text fixes a vulnerability.

See-also: https://go.dev/issue/56152
See-also: https://bugzilla.redhat.com/CVE-2022-32149
Signed-off-by: Niels de Vos <ndevos@redhat.com>
(cherry picked from commit e08005f402)
2022-10-18 14:16:08 +00:00
Rakshith R
f9adcde538 ci: fix mdl configuration
This commit makes the following changes:
`Please replace \":code_blocks => false\" \`
`by \":ignore_code_blocks => true\" in your configuration.`
Some rules are ignore for the time being,
these will be fixed later on.

Signed-off-by: Rakshith R <rar@redhat.com>
(cherry picked from commit b3837d44ce)
2022-10-18 10:34:15 +00:00
Madhu Rajanna
a3a0730900 rbd: return GRPC error message
The error message return from the GRPC
should be of GRPC error messages only
not the normal go errors. This commits
returns GRPC error if setAllMetadata
fails.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit 69eb6e40dc)
2022-10-18 10:21:06 +00:00
Madhu Rajanna
42bed5a346 rbd: delete volume if setallmetadata fails
If any operations fails after the volume creation
we will cleanup the omap objects, but it is missing
if setAllMetadata fails. This commits adds the code
to cleanup the rbd image if metadata operation fails.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit 01d4a614c3)
2022-10-18 10:21:06 +00:00
Madhu Rajanna
454bcc466a cephfs: use errors.As instead of errors.Is
As we need to compare the error type instead
of the error value we need to use errors.As
to check the API is implemented or not.

fixes: #3347

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit b40e8894f8)
2022-10-17 11:41:18 +00:00
Niels de Vos
fca90695fd e2e: disable rbd-nbd tests by default
Because the rbd-nbd tests fail with minikube and the Podman driver,
disable the tests for the time being.

Updates: #3431
Signed-off-by: Niels de Vos <ndevos@redhat.com>
(cherry picked from commit 386d3ddd6e)
2022-10-13 12:43:52 +00:00
Niels de Vos
ed44cdd621 e2e: log failures while deleting PVC and PV
There are occasions where deleting a PVC (or PV) never succeeds. The
reported status of the deleted object is sometimes empty, which suggests
that the PVC or PV was, in fact, deleted.

To diagnose the incorrect error checking, include the errors for
retrying in the logs.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
(cherry picked from commit 8eaf1d790d)
2022-10-13 12:43:52 +00:00
Niels de Vos
362471bd3b ci: fail installing Helm if wget is unavailable
In case `wget` is not installed, downloading the Helm release will fail.
The `install-helm.sh` script won't return a fatal error in that case,
and CI jobs continue running in an environment that is not ready.

By adding a check that exist the script with a failure, the CI will now
correctly report a problem when Helm can not be downloaded.

See-also: #3430
Signed-off-by: Niels de Vos <ndevos@redhat.com>
(cherry picked from commit fa97875dc9)
2022-10-13 12:43:52 +00:00
Niels de Vos
5635e498c0 e2e: wait for deployment before scale down/up
The scale down/up functions fail often with "deployment not found"
errors. Possibly deploying with Podman is slower than deploying in a
minikube VM, and there is a delay for the deployment to become
available.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
(cherry picked from commit 8f915576c4)
2022-10-13 12:43:52 +00:00
Niels de Vos
9d2d7da944 ci: add support for VM_DRIVER=podman to scripts/minikube.sh
When running on AWE EC2 virtual-machines, we'll use Podman instead of
installing a VM. The "none" driver might work as well, but it requires
additional dependencies to be installed, which may change over time with
new minikube or Kubernetes releases. Hopefully the Podman driver is less
affected with changes in dependencies.

Depends-on: #3419
Closes: #3415
Signed-off-by: Niels de Vos <ndevos@redhat.com>
(cherry picked from commit 0cba72485c)
2022-10-13 12:43:52 +00:00
Humble Chirammal
4aaf025713 Revert changes done for v3.7.1 release
This commit revert the changes made for 3.7.1 release back to -canary
tagging.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-09-30 20:49:20 +00:00
Humble Chirammal
a279a42975 helm: update image tag for release 3.7.1
This commit change the required image tag to
v3.7.1 instead of v3.7-canary for v3.7.1 release

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-09-16 12:44:16 +00:00
Humble Chirammal
f7cece7991 deploy: change image versions to v3.7.1
This commit change the required image tag to
v3.7.1 instead of v3.7-canary for v3.7.1 release

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-09-16 12:44:16 +00:00
Humble Chirammal
470d7e4183 doc: deprecate v3.5 as we are on v3.7
This commit mark v3.5 on deprecated release.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
(cherry picked from commit fc89159daf)
2022-09-14 05:03:17 +00:00
Humble Chirammal
93d5d96bbd doc: update doc for 3.7.1 release
updated doc for 3.7.1 release is available with this commit.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
(cherry picked from commit a799075cfc)
2022-09-14 05:03:17 +00:00
Rakshith R
acfe22efed rbd: use blocklist range cmd, fallback if it fails
This commit adds blocklist range cmd feature,
while fallbacks to old blocklist one ip at a
time if the cmd is invalid(not available).

Signed-off-by: Rakshith R <rar@redhat.com>
(cherry picked from commit a57859dfa4)
2022-09-13 13:55:17 +00:00
Prashanth Dintyala
e5e949d94a rbd: create token and use it for vault SA everytime possible
use TokenRequest API by default for vault SA even with K8s versions < 1.24

Signed-off-by: Prashanth Dintyala <vdintyala@nvidia.com>
(cherry picked from commit 2a6487cbf5)
2022-09-09 16:07:31 +00:00
Madhu Rajanna
4face6a7b3 cephfs: retry subvolumegroup creation
Incase the  subvolumegroup is deleted
and recreated we need to restart the
cephcsi provisioner pod to clear cache
that cephcsi maintains. With this PR
if cephcsi sees NotFound error duing
subvolume creation it will reset the cache
for that filesystem so that in next RPC
call cephcsi will try to create the
subvolumegroup again

Ref: https://github.com/rook/rook/issues/10623

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit 76064d8e34)
2022-09-08 11:37:05 +00:00
Madhu Rajanna
e08143a88b cephfs: fix subvolumegroup creation for multiple fs
In a cluster we can have multiple filesystem
for that we need to have a map of
subvolumegroups to check filesystem is created
nor not.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit e56621cd66)
2022-09-08 11:37:05 +00:00
Madhu Rajanna
468c73d2b6 ci: use resync to sync helm charts
When a file on source is deleted same
need to be deleted on the destination,
with rsync we can achieve it.

fixes: #3329

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit 9d46478794)
2022-09-07 11:30:25 +00:00
Madhu Rajanna
04879bbb33 rbd: map only primary image
If the image is mirroring enabled
and primary consider it for mapping,
if the image is mirroring enabled but
not primary yet. return error message
until the image is marked as primary.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit 71dbc7dbb4)
2022-09-06 14:05:08 +00:00
Humble Chirammal
66512b6a73 rbd: change default FsGroupPolicy to "File" for RBD CSI driver
This commit change the default fsgroup policy for csi driver object
to "File" type which is the better/correct setting for the CSI volumes.
We have been using default value which is "ReadWriteOnceWithFSType".
with this change backward compatibility should be preserved.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
(cherry picked from commit 586a9cc8ee)
2022-09-06 06:17:07 +00:00
Madhu Rajanna
3092b46774 cephfs: return success if metadata operation not supported
If the ceph cluster is of older version and doesnot
support metadata operation, Instead of failing
the request return the success if metadata
operation is not supported.

fixes #3347

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit 038462ff43)
2022-08-30 03:43:33 +00:00
Rakshith R
3a4c7c9d79 rbd: modify stripSecret mechanism in logGRPC()
This commit updates csi-addons spec version
and modifies logging to strip replication
request secret using csi.StripSecret, then
with replication.protosanitizer if the former
fails. This is done in order to make sure
we strip csi and replication format of secrets.

Signed-off-by: Rakshith R <rar@redhat.com>
(cherry picked from commit 40134772a7)
2022-08-29 14:55:24 +00:00
Rakshith R
2f393f24b7 rbd: improve kmip verifyResponse() error message
This commit uses %q instead %v in error messages
and adds result reason and message in kmip
verifyresponse().

Signed-off-by: Rakshith R <rar@redhat.com>
(cherry picked from commit f47839d73d)
2022-08-24 08:32:50 +00:00
Rakshith R
f3675f4f28 rbd: fix bug in kmip kms Decrypt function
This commit fixes a bug in kmip kms Decrypt
function, where emd.DEK was fed in a Nonce
instead of emd.Nonce by mistake.

Signed-off-by: Rakshith R <rar@redhat.com>
(cherry picked from commit eaa0e14cb2)
2022-08-24 08:32:50 +00:00
Humble Chirammal
0dde0e32b1 deploy: revert template changes of 3.7.0 to canary
Helm deployments and other deploy scripts are updated to have
canary tag in the release version

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-08-22 09:05:39 +00:00
Humble Chirammal
34fd27bbd1 deploy: change image versions to v3.7.0 instead of canary
This commit change the required image tag to release 3.7 instead
of canary for v3.7 release

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-08-19 13:25:11 +00:00
Humble Chirammal
aaa3d4e1d0 helm: update image tag for release 3.7 instead of canary
This commit change the image tag for release v3.7 instead of
canary.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-08-19 13:25:11 +00:00
Rakshith R
e4a280f06e doc: add nfs upgrade guide from 3.6 to 3.7
This commit adds nfs upgrade guide from 3.6
to 3.7.

Closes: #3271

Signed-off-by: Rakshith R <rar@redhat.com>
(cherry picked from commit 45f8232256)
2022-08-19 08:25:32 +00:00
Humble Chirammal
ff37d5fae8 rebase: use latest kubernetes version
This commit rebase the kubernetes version to 1.24.4 tree to make
sure we are using latest available bug fix release.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
(cherry picked from commit 2505c2ad14)
2022-08-19 13:08:25 +05:30
Humble Chirammal
efdc6b4572 doc: update documentation for release 3.7.0
This commit add upgrade documentation for release 3.7.0
and also update support matrix for v3.7.0.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
(cherry picked from commit 1e2a777d67)
2022-08-19 03:44:09 +00:00
Humble Chirammal
a9446c328e doc: correct e2e documentation formatting
This commit address formatting and alligment issues in e2e README.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-08-18 12:47:53 +00:00
Humble Chirammal
5cb8f80c10 build: enable ceph_pre_quincy tag
This commit add ceph_pre_quincy tag to the build. The main
reason being the changes or new APIs exposed in go-ceph version
v0.17.0 is part of this Ceph build tag. Enabling the same for
the reason.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-08-18 11:42:57 +00:00
Humble Chirammal
483181aec2 rebase: use v1.17.0 of go-ceph library
new version of go ceph is available and this commit make use
of the same.
Ref # https://github.com/ceph/go-ceph/releases/tag/v0.17.0

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-08-18 11:42:57 +00:00
Rakshith R
19e4146fab rbd: add replication capability & service to csiaddons server
csi-addons server will advertise replication capability and
replication service will run with csi-addons server too.

Signed-off-by: Rakshith R <rar@redhat.com>
2022-08-18 08:19:20 +00:00
Rakshith R
cb9ec35e3a rebase: update csi-addons/spec to latest
csi-addons/spec is updated to latest to
get replication identity capability.

Signed-off-by: Rakshith R <rar@redhat.com>
2022-08-18 08:19:20 +00:00
Rakshith R
e72ed593be rebase: vendor files required for kmip
Signed-off-by: Rakshith R <rar@redhat.com>
2022-08-18 07:41:42 +00:00
Rakshith R
0c33a33d5c rbd: add kmip encryption type
The Key Management Interoperability Protocol (KMIP)
is an extensible communication protocol
that defines message formats for the manipulation
of cryptographic keys on a key management server.
Ceph-CSI can now be configured to connect to
various KMS using KMIP for encrypting RBD volumes.

https://en.wikipedia.org/wiki/Key_Management_Interoperability_Protocol

Signed-off-by: Rakshith R <rar@redhat.com>
2022-08-18 07:41:42 +00:00
dependabot[bot]
2fc10ded65 rebase: bump github.com/aws/aws-sdk-go-v2/service/sts
Bumps [github.com/aws/aws-sdk-go-v2/service/sts](https://github.com/aws/aws-sdk-go-v2) from 1.16.9 to 1.16.13.
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go-v2/compare/v1.16.9...service/ivs/v1.16.13)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/service/sts
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-08-16 09:34:29 +00:00
Madhu Rajanna
842279b811 ci: use ubuntu-latest for stale job
Currently, we use the Ubuntu 18.04 actions runner
for stale job. This runner will be deprecated
and removed in the beginning of Dec.
So should change the runner to use latest ubuntu.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-08-16 07:00:44 +00:00
Madhu Rajanna
dde21543bd cephfs: fix staticcheck comment
getting is unused for linter "staticcheck"
(nolintlint) error message due to wrong
comment format. this the format now with
`//directive // comment`

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-08-10 17:51:26 +00:00
Madhu Rajanna
5c9f63bada ci: disable nosnakecase static check
All the CSI spec imports are having snakecase
we cannot fix it, instead of disabling all imports
skipping the nosnakecase static check.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-08-10 17:51:26 +00:00
Madhu Rajanna
a3751ada24 ci: update dependencies in build.env
This commits updates below items

* Golang version to v1.18.5
* Golangci-lint to v1.47.3
* Helm version to v3.9.2
* Rook version to v1.9.8

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-08-10 17:51:26 +00:00
Rakshith R
54fe4158a0 deploy: update fsGroupPolicy to File for nfs driver object
NFS volume similar to CephFs volumes can support
fsGroupPolicy as File, now Kubernetes Kubernetes may
use fsGroup to change permissions and ownership of the
volume to match user requested fsGroup in the pod's
SecurityPolicy regardless of fstype or access mode.

Signed-off-by: Rakshith R <rar@redhat.com>
2022-08-09 13:36:03 +00:00
Rakshith R
d39d2cffcc cleanup: use index instead of value while iterating
This commit cleans up for loop to use index to access
value instead of copying value into a new variable
while iterating.
```
internal/util/csiconfig.go:103:2: rangeValCopy: each \
iteration copies 136 bytes (consider pointers or indexing) \
(gocritic)
        for _, cluster := range config {
```

Signed-off-by: Rakshith R <rar@redhat.com>
2022-08-09 13:36:03 +00:00
Rakshith R
48d66d6cfd deploy: modify nfs daemonset to use cephcsi nfs nodeserver
This commit makes modification to nfs daemonset to use
nfs nodeserver. `nfs.NetNamespaceFilePath` example is
added.

Signed-off-by: Rakshith R <rar@redhat.com>
2022-08-09 13:36:03 +00:00
Rakshith R
3d3c029471 nfs: add nodeserver within cephcsi
This commit adds nfs nodeserver capable of
mounting nfs volumes, even with pod networking
using NSenter design similar to rbd and cephfs.
NodePublish, NodeUnpublish, NodeGetVolumeStats
and NodeGetCapabilities have been implemented.

The nodeserver implementation has been inspired
from https://github.com/kubernetes-csi/csi-driver-nfs,
which was previously used for mounted cephcsi exported
nfs volumes. The current implementation is also
backward compatible for the previously created
PVCs.

Signed-off-by: Rakshith R <rar@redhat.com>
2022-08-09 13:36:03 +00:00
Shyamsundar Ranganathan
c2280011d1 rbd: Report remote peer readiness if Up and status.Unknown
Current code uses an !A && !B condition incorrectly to
test A:Up and B:status for a remote peer image.

This should be !A || !B as we require both conditions to
be in the specified state (Up: true, and status Unknown).

This is corrected by this commit, and further fixes:
- check and return ready only when a remote site is
found in the status output
- check if all peer sites are ready, if multiple are found
and return ready appropriately

Signed-off-by: Shyamsundar Ranganathan <srangana@redhat.com>
2022-08-09 05:32:15 +00:00