1476 Commits

Author SHA1 Message Date
Marton Natko
d999d9db0c rbd: fix typo in createVolume validation
Signed-off-by: Marton Natko <mnatko@transip.nl>
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit 2fbca0d1e25b5ceb94df8525fda8336f404f8d9c)
2020-09-25 17:05:19 +00:00
Madhu Rajanna
764e33f249 cephfs: check ENOTEMPTY when removing subvolume
return a proper error message to the user when
the subvolume has the snapshots and it cannot
be removed until the snapshots on the subvolume
have to be deleted.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit 89b326c896a48393f7d99ab94445857b2bda7006)
2020-09-25 09:26:48 +00:00
Niels de Vos
cb7c3080cd ci: do not assume Docker is availble on the minikube host
Instead of using the Docker command to push the image to to minikube VM,
read the image from stdin over ssh and load it with the Docker command
that is available inside the VM.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-09-25 13:35:08 +05:30
Prasanna Kumar Kalever
ce0b3b4f88 ci: add a way to supply CONTAINER_CMD of choice
Add a way to supply local CONTAINER_CMD option of choice via
env variable to minikube.sh

Note: we still use docker daemon env at minikube box, in the future
we can switch to podman service env '# minikube podman-env' if needed

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2020-09-25 13:35:08 +05:30
Niels de Vos
54aed7f899 e2e: add simple ceph-fuse StorageClass tests
Signed-off-by: Niels de Vos <ndevos@redhat.com>
(cherry picked from commit f41cd105c46fcb6c9dc5fe3882a91d71f6ce6d0e)
2020-09-23 14:39:43 +00:00
Niels de Vos
126a71e51e e2e: pass SC parameters to createCephfsStorageClass()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
(cherry picked from commit 85df59531bde767645d05b67c8939b67178c80f4)
2020-09-23 14:39:43 +00:00
Niels de Vos
b810ca57a0 cephfs: trim spaces around fuseMountOptions
When passing

    fuseMountOptions: debug

in the StorageClass, the mount options passed on the ceph-fuse
commandline result in "-o nonempty ,debug". The additional space before
the ",debug" causes the mount command to fail.

Fixes: 1485
Signed-off-by: Niels de Vos <ndevos@redhat.com>
(cherry picked from commit 6f2f972f5b097d6d87654a890819b4bcfb101dc1)
2020-09-23 14:39:43 +00:00
Mudit Agarwal
bef4873049 build: update vendor with latest version
Updating the version of golang.org/x/text to version v0.3.3.
It fixes the vulnerability in http://golang.org/x/text/encoding/unicode
which leads UTF-16 decoder entering an infinite loop causing
the program to crash or run out of memory.

Signed-off-by: Mudit Agarwal <muagarwa@redhat.com>
(cherry picked from commit 8e434bb3ee2d173f451c09c1dbc8250b9547eccc)
2020-09-21 11:57:45 +00:00
Yug
8170e12114 cephfs: Fetch subvolume features in newSnapshotOptionsFromID
Get subvolume features using subvolume info and update the
volume options with Features parameter.

Signed-off-by: Yug <yuggupta27@gmail.com>
(cherry picked from commit 0682f0e50089cb5e5a099aeb866d02694aa3ff60)
2020-09-17 09:04:41 +00:00
Yug
6a64958876 cephfs: No-Op Unprotect call if "snapshot-autoprotect" feature present
The subvolume features consists the list of features, which if includes
"snapshot-autoprotect", result in query based approach to
detecting the need for protect/unprotect.
If "snapshot-autoprotect" feature is present, The UnprotectSnapshot
call should be treated as a no-op

Signed-off-by: Yug <yuggupta27@gmail.com>
(cherry picked from commit 9e959c45a388c0d4bc3edac24a35d89da5b0a453)
2020-09-17 09:04:41 +00:00
Yug
15463d8f52 cephfs: No-Op Protect call if "snapshot-autoprotect" feature present
The subvolume features consists the list of features, which if includes
"snapshot-autoprotect", result in query based approach to
detecting the need for protect/unprotect.
If "snapshot-autoprotect" feature is present, The ProtectSnapshot
call should be treated as a no-op

Signed-off-by: Yug <yuggupta27@gmail.com>
(cherry picked from commit 43a4c100ec73b588d47ba5902165bdaa6c314a8d)
2020-09-17 09:04:41 +00:00
Yug
b32569bc47 cephfs: use subvolume info if available
Use subvolume info to fetch the subvolume path.
If `subvolume info` command is not available,
use `getpath` command instead.

Signed-off-by: Yug <yuggupta27@gmail.com>
(cherry picked from commit 365eb58ff7134d3b738059684c447a5dc2209a0a)
2020-09-11 12:23:19 +00:00
Yug
0f51df99e8 cephfs: validate if subvolume has snapshot-retention feature
Add validateSnapshotRetention() to verify if subvolume has
`snapshot-retention` feature.

Signed-off-by: Yug <yuggupta27@gmail.com>
(cherry picked from commit 59fc4aa00ff25cc0917f307408ee454d4b97419a)
2020-09-11 12:23:19 +00:00
Yug
5daf21831e cephfs: Add --retain-snapshots flag to purge volume operations
Snapshots can be retained even after subvolume deletion in
Ceph 14.2.12. Adding support for the same in ceph-csi.

Signed-off-by: Yug <yuggupta27@gmail.com>
(cherry picked from commit 6351358752901b9c1c2e0f2ba797c256a73fd931)
2020-09-11 12:23:19 +00:00
Madhu Rajanna
cb09a3583e e2e: update cephfs ROX test case
updated cephfs ROX test case to create
ROX Clone PVC instead of creating an empty PVC

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit be3a78afaf97514cf3c1207c4c514a746add03ab)
2020-09-10 15:05:38 +00:00
Madhu Rajanna
d7e3f06384 cephfs: allow readonly only for volume with contentsource
It doesnot make sense to allow the creation of empty
volumes with readonly access, this commit allows the
creation of volume which is having readonly capabilities
only if the content source is set for the volume.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit 860b65c446f2b9c1ddc221c6f378c1d2f71fcbba)
2020-09-10 15:05:38 +00:00
Madhu Rajanna
5a7e188bb4 rbd: allow readonly only for volume with contentsource
It doesnot make sense to allow the creation of empty
volumes which is going to be accessed with readonly mode,
this commit allows the creation of volume which is having
readonly capabilities only if the content source is set
for the volume.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit 2deabcd2840ce06101c8646927f102ee1aeb947d)
2020-09-10 15:05:38 +00:00
Madhu Rajanna
c9601189ef util: add helper to validate ROX request
added a helper function to validate is create
request for ROX is for volume from source.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit d816df4f4226d88651cf14e0d8dbe2065437e7b3)
2020-09-10 15:05:38 +00:00
Madhu Rajanna
90da7961c5 e2e: rework on E2E framework
rework of E2E framework for better code
organization and add more helpful logs for
debugging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit b4693dcffeb4cb6ea69ed2d29bbea0bc1af094ec)
2020-09-10 08:54:20 +00:00
Mudit Agarwal
6105a4f428 rbd: NodeExpandVolume() should use StagingTargetPath
Form kubernetes v1.19 onwards NodeRequest is getting volume path
in StagingTargetPath instead of VolumePath, cephcsi should also
use the same.

Signed-off-by: Mudit Agarwal <muagarwa@redhat.com>
(cherry picked from commit 4de1abad5edaa074b9366ac551dd57657fc89d51)
2020-08-25 17:20:08 +00:00
Mudit Agarwal
1792cff878 rbd: implement rbd.updateVolWithImageInfo() with go-ceph
updateVolWithImageInfo() is currently executing an rbd command
to get details about an RBD image. Replaced it with the
required go-ceph functions.

Signed-off-by: Mudit Agarwal <muagarwa@redhat.com>
(cherry picked from commit 8c2bdbc0016b20959fac616c125ba84c29c65c23)
2020-08-25 08:13:52 +00:00
Madhu Rajanna
8e267b3806 rbd: rearrange the cleanup order incase clone fails
in case of clone failure, we need to first delete
the clone and the snapshot from which we created
the clone, then as part of cleanup we need to remove
the temporary cloned image and the temporary snapshot
created on the parent image.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit 38892c3068cfacf1f3b9595b681d30e31722ec2a)
2020-08-24 09:24:09 +00:00
Madhu Rajanna
45c8d178b2 rbd: fix return error message
incase of flattening error return the
flatten error.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit a7b65087af5ba98e3603073c09e21b1c6072b053)
2020-08-24 09:24:09 +00:00
Madhu Rajanna
8ab04d4160 e2e: add backend image validation for rbd
validate backend rbd images count in each
E2E test cases. This helps a lot to catch
the issues in each test case.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit d5c19954c1c7b586b15ce4b95a3a6344f446a6ba)
2020-08-24 09:24:09 +00:00
Madhu Rajanna
d18dd87de3 rebase: update go-ceph to v0.5.0
as go-ceph is 0.5.0 is released updating
the dependency to latest release.
more info about release at
https://github.com/ceph/go-ceph/releases/tag/v0.5.0

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit 2808d526bbeff302ba1d90c31d22d127a2c1cf9f)
2020-08-21 06:53:57 +00:00
Madhu Rajanna
8d1f3f09ba ci: change snapshotter default version to v2.1.1
we cannot depend on the master branch of external-snapshotter
in cephcsi as the master branch can change anytime. its
good to use released tags to our E2E.

fixes: #1416

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit d77b23e3c199e25b05437b618212001da251a560)
2020-08-20 06:47:49 +00:00
Madhu Rajanna
a8e6bdee8e cephfs: use string const for cli error comparison
we should not return the CLI errors in GRPC errors
we need to return proper readable error messages
to the user for better understanding and better
debugging.

updates #1242

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit 0bd45a5eb172477ea29e3b9c97eb79b654587044)
2020-08-19 18:15:10 +00:00
Madhu Rajanna
b2a9ecf5e5 util: replace klog with util logger in liveness.go
replace klog with util logger in liveness.go

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit 27d2cc644de2fbcaa182006faafe03ef3565da3c)
2020-08-19 15:35:13 +00:00
Madhu Rajanna
2fe86bbc4b journal: replace klog with util logger in voljournal.go
journal: replace klog with util logger in voljournal.go

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit ed289fea5e44a84a9a8840846d3e8d20499fa781)
2020-08-19 15:35:13 +00:00
Madhu Rajanna
c75f89628c journal: replace klog with util logger in omap.go
replace klog with util logger in omap.go

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit 1a9cd23f6475522cc9c7b33c93e57c31bca06085)
2020-08-19 15:35:13 +00:00
Madhu Rajanna
86aca6c631 rbd: replace klog with util logger in snapshot.go
rbd: replace klog with util logger in snapshot.go

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit 7c9fee17af7d564bd995f610893ae456ab8799c0)
2020-08-19 15:35:13 +00:00
Madhu Rajanna
c61fd94596 rbd: replace klog with util logger in rbd_util.go
rbd: replace klog with util logger in rbd_util.go

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit 9305e277972510e130a36f0c4fb4074c331bec55)
2020-08-19 15:35:13 +00:00
Madhu Rajanna
9938c654e1 rbd: replace klog with util logger in rbd_journal.go
replace klog with util logger in rbd_journal.go

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit 90cc5a4fd0a0f077294135c86e71cae70128f119)
2020-08-19 15:35:13 +00:00
Madhu Rajanna
f9bb13a373 rbd: replace klog with util logger in rbd_attach.go
replace klog with util logger in rbd_attach.go

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit 9512aa6206cfa03d7c177d1432bd503463855b23)
2020-08-19 15:35:13 +00:00
Madhu Rajanna
727666dedc rbd: replace klog with util logger in nodeserver.go
replace klog with util logger in nodeserver.go

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit e768c0dfc01c7d19a92189fde7e42a2fdcc963ad)
2020-08-19 15:35:13 +00:00
Madhu Rajanna
4b5952d15a rbd: replace klog with util logger in driver.go
replace klog with util logger in driver.go

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit 883485b082f57c2adaf8286abf3e4ffa56b4461e)
2020-08-19 15:35:13 +00:00
Madhu Rajanna
ab53834757 rbd: replace klog with util logger in clone.go
replace klog with util logger in clone.go

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit c4f30c23c1f4b6dc908ae230d87f6a35bc9f2d88)
2020-08-19 15:35:13 +00:00
Madhu Rajanna
98088ec50a rbd: replace klog with util in controllerserver.go
replace klog with util in controllerserver.go

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit d8ca56fbe9dfe4d4f5fac43d265b9938f1eda04e)
2020-08-19 15:35:13 +00:00
Humble Chirammal
d04dbd0811 rbd: Bail out from nodeexpansion if its block mode pvc
At CSI spec < 1.2.0, there was no volumecapability in the
expand request. However its available from v1.2+ which allows
us to declare the node operations based on the volume mode.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
(cherry picked from commit 1f5b84745facc983936ed2004f2bedc802fcec20)
2020-08-19 13:43:52 +00:00
Madhu Rajanna
165b8d13d1 cephfs: fix lock release in volumeDelete
currently the lock is not released which is
taken on the request name. this is causing issues
when the subvolume is requested for delete.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit 787ba690894b46b3b270f6683efac2ea33391b91)
2020-08-18 10:07:25 +00:00
Madhu Rajanna
f9faf4822f cephfs: check error output contains Error: ENOENT
execCommandErr returns both error and stderror
message. checking strings.HasPrefix is not helpful
as the stderr will be the first string. its good
to do string comparison and find out that error
is volume not found error.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
(cherry picked from commit 31c7c95fdbabb4e43bcf7551cd0b8a596d94b606)
2020-08-18 10:07:25 +00:00
Madhu Rajanna
91475c0a66 revert: revert back the changes for v3.1.0
as the release is done this commit reverts
back the changes to build canary images
from release branch.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-17 08:28:32 +05:30
Humble Chirammal
5d48473582 build: update templates for v3.1.0 release
updated required templates for v3.1.0 release.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
v3.1.0
2020-08-13 21:04:51 +05:30
Humble Chirammal
7111f03cf2 ci: update Makefile and deploy.sh for image tag
as we need to build and push the v3.1.0 image,
updated the Makefile and deploy.sh to build and
push image and helm tempalates

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-13 21:04:51 +05:30
Humble Chirammal
a11c370b72 ci: update branch name in the travis CI
The branch name has to be updated in the
travis yaml to run the tests and upload the
templates and release image.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-13 21:04:51 +05:30
Madhu Rajanna
f11486f4b6 doc: add note for cephfs snapshot limitation
added a note for cephfs snapshot limitations

closes #1242

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 11:55:23 +00:00
Madhu Rajanna
fabe84d4fe doc: add upgrade documentation for v3.1.0
added upgrade documentation to upgrade from
v3.0.0 to v3.1.0

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 11:03:36 +00:00
Humble Chirammal
1b94ed2b37 doc: update cephfs snapshot, restore and clone support matrix
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-13 10:12:14 +00:00
Madhu Rajanna
f566b0fc2d cephfs: replace Warningf with WarningLog in unmountVolume
replace Errorf with ErrorLog in unmountVolume
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
3fe3d25dda cephfs: replace Errorf with ErrorLogMsg in loadAvailableMounters
replace Errorf with ErrorLog in loadAvailableMounters
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00