1
0
mirror of https://github.com/ceph/ceph-csi.git synced 2024-12-22 21:10:22 +00:00
Commit Graph

331 Commits

Author SHA1 Message Date
Robert Vasek
80dda7cc30 cephfs: detect corrupt ceph-fuse mounts and try to remount
Mounts managed by ceph-fuse may get corrupted by e.g. the ceph-fuse process
exiting abruptly, or its parent container being terminated, taking down its
child processes with it.

This commit adds checks to NodeStageVolume and NodePublishVolume procedures
to detect whether a mountpoint in staging_target_path and/or target_path is
corrupted, and remount is performed if corruption is detected.

Signed-off-by: Robert Vasek <robert.vasek@cern.ch>
2022-03-10 06:05:52 +00:00
Robert Vasek
aa6297e164 cleanup: refactor helper functions in nodeserver.go
Refactored a couple of helper functions for easier resue.

* Code for building store.VolumeOptions is factored out into a separate function.

* Changed args of getCredentailsForVolume() and NodeServer.mount() so that
  instead of passing in whole csi.NodeStageVolumeRequest, only necessary
  properties are passed explicitly. This is to allow these functions to be
  called outside of NodeStageVolume() where NodeStageVolumeRequest is not
  available.

Signed-off-by: Robert Vasek <robert.vasek@cern.ch>
2022-03-10 06:05:52 +00:00
Madhu Rajanna
e9802c4940 cephfs: refactor cephfs core functions
This commits refactors the cephfs core
functions with interfaces. This helps in
better code structuring and writing the
unit test cases.

update 

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-02-22 20:39:23 +00:00
Humble Chirammal
8f6a7da538 cephfs: dont set explicit permissions on the volume
At present we are node staging with worldwide permissions which is
not correct. We should allow the CO to take care of it and make
the decision. This commit also remove `fuseMountOptions` and
`KernelMountOptions` as they are no longer needed

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-02-09 17:30:29 +00:00
Madhu Rajanna
2943555904 cephfs: fix omap deletion in DeleteSnapshot
the omap is stored with the requested
snapshot name not with the subvolume
snapshotname. This fix uses the correct
snapshot request name to cleanup the omap
once the subvolume snapshot is deleted.

fixes: 

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-02-08 20:37:53 +00:00
Madhu Rajanna
992d257530 cephfs: fix error logging in filesystem.go
fix error message logging in filesystem.go

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-01-27 14:31:12 +00:00
Madhu Rajanna
14c008c419 cleanup: use interface in filesystem.go
Currently, we are using methods and all the methods
makes a network call to fetch details from the ceph
clusters, its difficult to write test cases for
these functions, if we move to the interfaces
we can make use of mock to write unit testing
for the caller functions.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-01-27 14:31:12 +00:00
Madhu Rajanna
2daf2f9f0c cephfs: log error message if clone fails
During CreateVolume from snapshot/volume,
its difficult to identify if the clone is
failed and a new clone is created. In case
of clone failure logging the error message
for better debugging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-01-17 09:43:09 +00:00
Madhu Rajanna
ef14ea7723 cephfs: resize cloned, restored volume if required
Currently, as a workaround, we are calling
the resize volume on the cloned, restore volumes
to adjust the cloned, restored volumes.
With this fix, we are calling the resize volume
only if there is a size mismatch with requested
and the volume from which the new volume needs
to be created.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2022-01-12 10:44:11 +00:00
Humble Chirammal
68350e8815 cephfs: add SINGLE_NODE_{SINGLE/MULTI}_WRITER capability
SINGLE_NODE_WRITER capability ambiguity has been fixed in csi spec v1.5
which allows the SP drivers to declare more granular WRITE capability.
These are not really new capabilities rather capabilities introduced to
get the desired functionality from CO side based on the capabilities SP
driver support for various CSI operations.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2022-01-11 19:40:22 +00:00
Madhu Rajanna
12e8e46bcf revert: remove explicit size setting of cloned volume
The ceph changes  are done on the both server and the
client side this change is not enough for remove
setting the size of cloned volumes.
this caused the regression like    .

This reverts commit 3565a342d5.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-12-21 14:15:46 +00:00
Madhu Rajanna
34a21cdbe3 cleanup: move mount functions to new pkg
moved fuse and kernel mount functions
to a new package.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-09-23 06:39:37 +00:00
Madhu Rajanna
b1ef842640 cleanup: move core functions to core pkg
as we are refractoring the cephfs code,
Moving all the core functions to a new folder
/pkg called core. This will make things easier
to implement. For now onwards all the core
functionalities will be added to the core
package.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-09-23 06:39:37 +00:00
Rakshith R
82d09d81cf util: modify GetMonsAndClusterID() to take clusterID instead of options
This commit:
- modifies GetMonsAndClusterID() to take clusterID instead of options.
- moves out validation of clusterID is set or not out of GetMonsAndClusterID().
- defines ErrClusterIDNotSet new error for reusability.
- add GetClusterID() to obtain clusterID from options.

Signed-off-by: Rakshith R <rar@redhat.com>
2021-09-14 08:39:57 +00:00
Rakshith R
9d1e98ca60 rbd: check for clusterid mapping in genVolFromVolumeOptions()
This commit adds capability to genVolFromVolumeOptions() to fetch
mapped clusted-id & mon ips for mirrored PVC on secondary cluster
which may have different cluster-id.

This is required for NodeStageVolume().

We also don't need to check for mapping during volume create requests,
so it can be disabled by passing a bool checkClusterIDMapping as false.

GetMonsAndClusterID() is modified to accept bool checkClusterIDMapping
based on which clustermapping is checked to fetch mapped cluster-id and
mon-ips.

Signed-off-by: Rakshith R <rar@redhat.com>
2021-09-14 08:39:57 +00:00
Rakshith R
0a7a7f4866 util: call WriteCephConfig() in cephcsi.go
This commit calls WriteCephConfig() in cephcsi.go to
create ceph.conf and keyring if it is not mounted to
be used by all cli calls and conn cmds.

Before this change, rbd-controller/omap-generator did not create
ceph.conf on startup.

Signed-off-by: Rakshith R <rar@redhat.com>
2021-09-08 16:05:27 +00:00
Madhu Rajanna
76f1b42498 cephfs: correct comment for validateExpandVolumeRequest
corrected the function comment for
validateExpandVolumeRequest.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-09-07 14:33:02 +00:00
Madhu Rajanna
9fd51d9bec cephfs: add comment for validateCreateVolumeRequest
added function comment for
validateCreateVolumeRequest

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-09-07 14:33:02 +00:00
Madhu Rajanna
8caeb409bb cephfs: add comment for validateDeleteVolumeRequest
added function comment for the
validateDeleteVolumeRequest function.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-09-07 14:33:02 +00:00
Madhu Rajanna
be7749c90e cleanup: move volumeID to the volumeoptions
volumeID can be moved to the volumeOptions
as most of the volume related helper functions
are available on the volumeoptions.go

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-09-07 14:33:02 +00:00
Madhu Rajanna
da70ed50dc cleanup: move execCommandErr to volumemounter
Moved execCommandErr to the volumemounter.go
which is the only caller of this function and
moving the execCommandErr helps in reducing the
util file.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-09-07 14:33:02 +00:00
Madhu Rajanna
31696a6ce0 cleanup: move genSnapFromOptions to volumeoptions
moved genSnapFromOptions function to volumeoptions.go
which is more appropriated than util.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-09-07 14:33:02 +00:00
Madhu Rajanna
73e2ffe8b8 cleanup: move cephfs csi spec validation to validator
moved the cephfs related validation like
validating the input parameters sent in the
GRPC request to a new file.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-09-07 14:33:02 +00:00
Humble Chirammal
df2d9548ae cephfs: no need to check for zero volume size
At present there is a 'todo' to check for zero volume size
in the createVolume request which in unwanted, ie the pvc
creation with size 0 fail from the kubernetes api validation itself:

For ex:

```
..spec.resources[storage]: Invalid value: "0": must be greater than zero```
```
so we dont need any extra check in the controller server

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-09-07 04:49:24 +00:00
Humble Chirammal
247795517f cephfs: remove explicit size setting of cloned volume
CephFS csi driver explictly set the size of the cloned volume
to the size of parent volume as cephfs mgr was lacking this
functionality previously. However it has been addressed in cephfs
so we dont need explicit size setting.

Ref#https://tracker.ceph.com/issues/46163

Supported Ceph releases:

Ceph versions equal or above - v16.0.0, v15.2.9, v14.2.12

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-09-01 09:32:29 +00:00
Madhu Rajanna
b383af20b4 cleanup: move cephfs errors to new util package
As part of the refactoring, moving the cephfs errors file to a new
package.

Updates: 
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-09-01 06:50:16 +00:00
Niels de Vos
6d00b39886 cleanup: move log functions to new internal/util/log package
Moving the log functions into its own internal/util/log package makes it
possible to split out the humongous internal/util packages in further
smaller pieces. This reduces the inter-dependencies between utility
functions and components, preventing circular dependencies which are not
allowed in Go.

Updates: 
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-08-26 09:34:05 +00:00
Humble Chirammal
edf511a833 cephfs: make use of subvolumeInfo.state to determine quota
https://github.com/ceph/go-ceph/pull/455/ added `state` field
to subvolume info struct which helps to identify the snapshot
retention state in the caller. This patch make use of the same

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-08-18 04:50:46 +00:00
Humble Chirammal
66fa5891b2 cephfs: correct typos in cephfs driver code
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-08-18 04:50:46 +00:00
Alexandre Lossent
5cba04c470 cephfs: support selinux mount options
- mount host's /etc/selinux in node plugins
- process mount options in all code paths for cephfs volume options

Signed-off-by: Alexandre Lossent <alexandre.lossent@cern.ch>
2021-08-04 12:59:34 +00:00
Yati Padia
1ae2afe208 cleanup: modifies the error caused due to merged PRs
This commit modifies the error of godot, cyclop,
paralleltest linter caused due to merged PRs.

Updates: 

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-07-22 18:15:48 +00:00
Yati Padia
172b66f73f cleanup: resolves cyclop linter issue
this commit adds `// nolint:cyclop` for the
fucntions whose complexity is above 20

Updates: 

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-07-22 18:15:48 +00:00
Rakshith R
43f753760b cleanup: resolve nlreturn linter issues
nlreturn linter requires a new line before return
and branch statements except when the return is alone
inside a statement group (such as an if statement) to
increase code clarity. This commit addresses such issues.

Updates: 

Signed-off-by: Rakshith R <rar@redhat.com>
2021-07-22 06:05:01 +00:00
Humble Chirammal
e088e8fd2e cephfs: Get rid of locking at nodepublish
Considering kubelet make sure the stage and publish operations
are serialized, we dont need any extra locking in nodePublish

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-07-16 07:18:56 +00:00
Humble Chirammal
ced3a0922f cephfs: Get rid of locking at nodeUnpublish call
Considering kubelet make sure the unstage and unpublish operations
are serialized, we dont need any extra locking in nodeUnpublish

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-07-16 07:18:56 +00:00
Yati Padia
f36d611ef9 cleanup: resolves gofumpt issues of internal codes
This PR runs gofumpt for internal folder.

Updates: 

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-07-14 19:50:56 +00:00
Yati Padia
299979fc14 ci: add unit test for toError()
This commit adds unit test for the
func converting cephFSCloneState to error.

Fixes: 

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-07-14 15:02:12 +00:00
Yati Padia
c66872c3c6 cleanup: ineffective assignment
This commit resolves ineffective assignent of
snap.

Updates: 

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-07-14 12:39:17 +00:00
Yati Padia
4a649fe17f cleanup: resolve godot linter
This commit resolves godot linter issue
which says "Comment should end in a period (godot)".

Updates: 

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-07-13 06:50:03 +00:00
Yati Padia
84c1fe52c7 cleanup: resolve exhaustive linter
This commit resolves exhaustive linter
error.

Updates: 

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-07-12 04:47:08 +00:00
Humble Chirammal
0d432be5bf internal: reformat long lines in internal/cephfs package to 120 chars
We have many declarations and invocations..etc
with long lines which are very difficult to follow while doing
code reading. This address the issues in 'internal/cephfs' package to
restrict the line length to 120 chars.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-06-28 14:43:49 +00:00
Madhu Rajanna
4e2c4ef704 cephfs: return internal server error
if it is an error from the IsMountPoint
function and the error is not IsNotExist return
it as a internal server error.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-06-07 07:38:48 +00:00
Madhu Rajanna
46f1ab9e99 cephfs: use IsMountPoint to check mountpoint
Currently we are relaying on the error output from
the umount command we run on the nodes when mounting
the volume but we are not checking for all the error
message to verify the volume is mounted or not.
This commits uses IsMountPoint function in util
to check the mountpoint.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-06-07 07:38:48 +00:00
YingshuoTao
bfe64d4aee cephfs: pass extra volume attributes to static PV
when using pre-provisioned volumes, pass these parameters:
- kernelMountOptions
- fuseMountOptions
- subVolumeGroup
in spec.csi.volumeAttributes in PV declaration

Signed-off-by: YingshuoTao <frigid.blues@gmail.com>
2021-06-03 04:42:59 +00:00
Madhu Rajanna
a666d452bf cephfs: return GRPC error in NodeGetVolumeStats
in case of failure return GRPC error to the caller.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-05-31 08:17:37 +00:00
Yati Padia
774e8e4042 util: enable golang profiling
Add support for golang profiling.
Standard tools like go tool pprof and curl
work. example:
$ go tool pprof http://localhost:8080/debug/pprof/profile
$ go tool pprof http://localhost:8080/debug/pprof/heap
$ curl http://localhost:8080/debug/pprof/heap?debug=1

https://golang.org/pkg/net/http/pprof/ contains
more details about the pprof interface.

Fixes: 

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-05-25 10:41:22 +00:00
Niels de Vos
c0ab4c03e6 cephfs: move NodeGetVolumeStats() to CephFS NodeServer
The CephFS NodeServer should handle the CephFS specific requests. This
is not something that the NodeServer for RBD should handle.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-05-25 06:41:04 +00:00
Mudit Agarwal
ec105bd782 cephfs: expand clone error messages
Adding "snapshot clone" in the clone error messages.

Signed-off-by: Mudit Agarwal <muagarwa@redhat.com>
2021-04-26 13:38:55 +00:00
Humble Chirammal
0fae0e53b6 cleanup: various source code comment corrections
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-04-16 10:22:35 +00:00
Rakshith R
10d539efc8 cleanup: correct nolint directive listing format
nolint directive needs to be followed by comma separated
list of linters. This commit changes to gocognit:gocyclo
which was not recognised to linters which show error for
the function.

Signed-off-by: Rakshith R <rar@redhat.com>
2021-04-09 07:24:47 +00:00
Rakshith R
fb7389f478 cephfs: add stderr to mount function errors
This commit appends stderr to error in both kernel and
ceph-fuse mounter functions to better be able to debug
errors.

Signed-off-by: Rakshith R <rar@redhat.com>
2021-04-08 12:18:01 +00:00
Rakshith R
2d1a572d11 cleanup: refactor deeply nested if statements in internal/cephfs
Refactored deeply nested if statement in internal/cephfs to
reduce cognitive complexity.

Signed-off-by: Rakshith R <rar@redhat.com>
2021-04-07 02:31:41 +00:00
Rakshith R
721640178b cephfs: fix unmountVolume function
This commit fixes bug in unmount function which caused
unmountVolume to fail when targetPath was already unmounted.

Signed-off-by: Rakshith R <rar@redhat.com>
2021-03-25 15:15:07 +00:00
Yati Padia
0d9548c815 Cephfs: Failed to delete snapshot
Failed to delete voluesnapshot when backend subvolume
(pvc) and ceph fs subvolume snapshot is deleted

Fixes#1647

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-03-17 10:28:08 +00:00
Madhu Rajanna
c642637cec util: register replication controller
as RBD is implementing the replication
we are registering it. For CephFS, its
not implementing the replication we are
passing nil so we dont want to register
it.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-03-16 13:06:44 +00:00
Madhu Rajanna
ee0576278f cleanup: move servers to a new struct
For future enhancments like adding more
servers. Moving the list of servers to a
new structure.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-03-16 13:06:44 +00:00
Madhu Rajanna
8720f4e2f5 cephfs: create subvolume with VolumeNamePrefix
when user provides an option for VolumeNamePrefix
create subvolume with the prefix which will be easy
for user to identify the subvolumes belongs to
the storageclass.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-02-19 17:04:17 +00:00
Madhu Rajanna
8cd901d2dd cephfs: add subvolume path to volume context
There are many usecases with adding the subvolume
path to the PV object. the volume context returned
in the createVolumeResponse is added to the PV object
by the external provisioner.

More Details about the usecases are in below link
https://github.com/rook/rook/issues/5471

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-02-11 11:31:22 +00:00
Mudit Agarwal
d480eb4bda cephfs: ignore BytesQuota field in case it is not set.
This can happen when the subvolume is in snapshot-retained state.
We should not return error for such case as it is a valid situation.

Signed-off-by: Mudit Agarwal <muagarwa@redhat.com>
2021-02-01 09:20:53 +00:00
Yati Padia
995879d349 cephfs: use enum to check resize is supported or not
Currently, we are using bool pointer to find out
the ceph cluster supports resize or not. This
commit replaces the bool pointer with enum.

Signed-off-by: Yati Padia <ypadia@redhat.com>

Fixes#1764
2021-01-04 04:42:58 +00:00
Madhu Rajanna
0d9fcbc21b cephfs: remove unused cr Credentials
removed unused cr variable.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-10 10:13:55 +00:00
Madhu Rajanna
9a96370942 cephfs: implement purgeVolume with go-ceph
moved frm  ceph fs CLI to go-ceph for
purgeVolume.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-10 10:13:55 +00:00
Madhu Rajanna
34d0ff0d70 cephfs: make purgeVolume method of volumeOptions
converted purgeVolume from function to method
of volumeOptions.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-10 10:13:55 +00:00
Niels de Vos
16cb43f0f9 rbd: store csi.storage.k8s.io/pvc/namespace metadata as Owner
The Owner of an RBD image (Kubernetes Namespace, tenant) can be used to
identify additional configuration options. This will be used for
fetching the right Vault Token when encrypting/decrypting volumes.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-09 13:58:48 +00:00
Niels de Vos
a7a928d7ec cleanup: return error from execCommandErr() in bindMount()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-09 08:35:35 +00:00
Madhu Rajanna
ebb413534f cephfs: remove unsed cr util.Credentials
remove unused cr util.Credentials variable.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-03 14:53:33 +00:00
Madhu Rajanna
0f451ed465 cephfs: implement getSnapshotInfo with go-ceph
implement getSnapshotInfo function with go-ceph
to get subvolume information.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-03 14:53:33 +00:00
Madhu Rajanna
ddf91de859 cephfs: implement clonesnapshot with go-ceph
updated ceph fs CLI implementaion of cloning
with go-ceph.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-03 06:39:18 +00:00
Madhu Rajanna
814bf4459a cephfs: implement snapshot protect and unprotect to go-ceph
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-03 06:39:18 +00:00
Niels de Vos
3433cefaa5 cleanup: standardize logging in deleteSnapshot
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-02 14:36:39 +00:00
Niels de Vos
294f7b22d4 cephfs: remove unused Credentials argument from deleteSnapshot()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-02 14:36:39 +00:00
Niels de Vos
9d9b5b3303 cephfs: implement deleteSnapshot() with go-ceph
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-02 14:36:39 +00:00
Niels de Vos
032db78d8a cephfs: make deleteSnapshot() a method of volumeOptions
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-02 14:36:39 +00:00
Niels de Vos
5883f244d2 cleanup: standardize logging in createSnapshot
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-02 13:18:47 +05:30
Niels de Vos
7c580af412 cephfs: remove unused Credentials argument from createSnapshot()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-02 13:18:47 +05:30
Niels de Vos
1d8cc75282 cephfs: implement createSnapshot() with go-ceph
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-02 13:18:47 +05:30
Niels de Vos
57f8ee0aa7 cephfs: make createSnapshot() a method of volumeOptions
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-02 13:18:47 +05:30
Madhu Rajanna
a35a835e9c cephfs: fix logic to call ceph fs resize
clusterAdditionalInfo map is holding a localClusterState
for checking ceph cluster supports resize and subvolumegroup
is created or not, currently we are checking if the key is present
in a map and localClusterStatelocalClusterState.resizeSupported
is set to false to call ceph fs subvolume resize to check command is
supported or not, if a structure is  initialized all its members
are set to default value. so we will never going to check the
ceph fs subvolume  resize command is supported in backend or not, we are
always using ceph fs subvolume create to resize subvolume. in some
ceph version ceph fs subvolume create wont work to resize a subvolume.
This commit changes the resizeSupported from bool to *bool for
proper handling of this scenario.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-01 13:31:39 +00:00
Madhu Rajanna
a1c8942e2b cephfs: remove unused invalidCommand error const
removed unused invalidCommand from cephfs
error const.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-01 09:46:52 +00:00
Madhu Rajanna
1ba160afdb cephfs: handle invalid error in subvolume info
go ceph returns NotImplementedError for invalid
commands,cephcsi is using errors.As to find out
the error.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-01 09:46:52 +00:00
Madhu Rajanna
66eb3b7525 cephfs: handle invalid error in fs resize
go ceph returns NotImplementedError for invalid
commands,cephcsi is using errors.As to find out
the error.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-01 09:46:52 +00:00
Madhu Rajanna
2232067e99 cephfs: use rados.ErrNotFound in subvolume info
go-ceph rados.ErrNotFound for not found errors,cephcsi
need to check same for not found errors.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-01 09:46:52 +00:00
Madhu Rajanna
a0283ef7f9 cephfs: use rados.ErrNotFound in getVolumeRootPathCeph
go-ceph rados.ErrNotFound for not found errors,cephcsi
need to check same for not found errors.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-01 09:46:52 +00:00
Madhu Rajanna
b40d561228 cephfs: make getVolumeRootPathCeph as a method
getVolumeRootPathCeph is a method of volumeOptions

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-01 09:46:52 +00:00
Madhu Rajanna
e4713bc9ed cephfs: remove unused Credentials argument from getVolumeRootPathCeph
remove unused Credentials argument from getVolumeRootPathCeph
function.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-01 09:46:52 +00:00
Madhu Rajanna
b6f3ba885e cephfs: implement getVolumeRootPathCeph with go-ceph
instead of ceph fs CLI commands using go-ceph library
to get subvolume path.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-01 09:46:52 +00:00
Madhu Rajanna
39b1f2b4d3 cleanup: fix mispell words
fixed mispell words in the repo.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-11-29 12:47:46 +05:30
Niels de Vos
8d08e8b37e cephfs: remove unused Credentials argument from getCloneState()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-29 07:13:35 +00:00
Niels de Vos
055975f790 cephfs: implement getCloneState() with go-ceph
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-29 07:13:35 +00:00
Niels de Vos
476ad18d2e cephfs: make getCloneState() a method of volumeOptions
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-29 07:13:35 +00:00
Niels de Vos
66aa595e77 cephfs: do not export internal CloneStatus type
getCloneInfo() does not need to return a full CloneStatus struct that
only has one member. Instead, it can just return the value of the single
member, so the JSON type/struct does not need to be exposed.

This makes the API for getCloneInfo() a little simpler, so it can be
replaced by a go-ceph implementation later on.

As the function does not return any of the unused attributes anymore, it
is renamed to getCloneStatu() as well.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-29 07:13:35 +00:00
Madhu Rajanna
ba84f14241 journal: create object with provided UUID
incase of async mirroring the volume UUID is
retrieved from the volume name, instead of cephcsi
generating a new UUID it should reserve the passed
UUID it will be useful when we support both metro DR
and async mirroring on a kubernetes clusters.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-11-28 18:50:00 +00:00
Yug
3ac6bbd87c cephfs: Add isCloneRetryError function
The function isCloneRetryError verifies
if the clone error is `pending` or
`in-progress` error.

Co-authored-by: Madhu Rajanna <madhupr007@gmail.com>
Signed-off-by: Yug <yuggupta27@gmail.com>
2020-11-09 07:29:12 +00:00
Yug
acbedc52bf cephfs: Add 'pending' state for clone status
In certain cases, clone status can be 'pending'.
In that case, abort error message should be
returned similar to that during 'in-progress'
state.

Co-authored-by: Madhu Rajanna <madhupr007@gmail.com>
Signed-off-by: Yug <yuggupta27@gmail.com>
2020-11-09 07:29:12 +00:00
Niels de Vos
565038fdfd cephfs: ignore quota when SubVolumeInfo() returns Infinite
There is a type-check on BytesQuota after calling SubVolumeInfo() to see
if the value is supported. In case no quota is configured, the value
Infinite is returned. This can not be converted to an int64, so the
original code returned an error.

It seems that attaching/mounting sometimes fails with the following
error:

    FailedMount: MountVolume.MountDevice failed for volume "pvc-0e8fdd18-873b-4420-bd27-fa6c02a49496" : rpc error: code = Internal desc = subvolume csi-vol-0d68d71a-1f5f-11eb-96d2-0242ac110012 has unsupported quota: infinite

By ignoring the quota of Infinite, and not setting a quota in the
Subvolume object, this problem should not happen again.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-06 14:58:26 +00:00
Niels de Vos
9732cf16a1 cephfs: drop unused Credentials from resizeVolume()
When using go-ceph and the volumeOptions.Connect() call, the credentials
are not needed once the connection is established.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-28 08:02:12 +00:00
Niels de Vos
5baed6190c cephfs: implement resizeVolume() with go-ceph
Reduce the number of calls to the `ceph fs` executable to improve
performance of CephFS volume resizing.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-28 08:02:12 +00:00
Niels de Vos
d431402101 cephfs: make resizeVolume() a method of volumeOptions
This prepares resizeVolume() so that the volumeOptions.conn can be used
for connecting with go-ceph and use the connection cache.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-28 08:02:12 +00:00
Niels de Vos
6a46b8f17f cephfs: implement getSubVolumeInfo() with go-ceph
Fixes: 
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-23 10:58:35 +00:00
Niels de Vos
429f7acd8f cephfs: make getSubVolumeInfo() a method of volumeOptions
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-23 10:58:35 +00:00
Niels de Vos
5a1e370433 util: drop nolint comment for execCommandJSON()
golang-ci suddenly complains about the following issue

    internal/cephfs/util.go:41:1: directive `// nolint:unparam //  todo:program values has to be revisited later` is unused for linter unparam (nolintlint)
    // nolint:unparam //  todo:program values has to be revisited later
    ^

Dropping the comment completely seems to fix it. Ideally
execCommandJSON() will get removed once the migration to go-ceph is
complete.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-20 15:38:44 +00:00
Niels de Vos
eb2584095b cephfs: implement getMetadataPool() with go-ceph
Fixes: 
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-20 15:38:44 +00:00
Niels de Vos
cabdac4913 cephfs: make getMetadataPool() a method of volumeOptions
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-20 15:38:44 +00:00
Humble Chirammal
9dee064b77 cephfs: fix wrong error check in CreateVolume rollback action
Previously the purgeVolume error was ignored due to wrong error variable
check in the createVolume. With this change it checks on the proper error.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-10-20 04:08:51 +00:00
Humble Chirammal
5b537754a7 cephfs: replace variable name for builtin cap attribute
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-10-19 13:25:15 +00:00
Niels de Vos
0e96199a1c cephfs: free resources when newVolumeOptionsFromVolID() errors out
The allocated, and potentially connected, volumeOptions object in
newVolumeOptionsFromVolID() is not cleaned-up in case of errors. This
could cause resource leaks.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-19 09:15:43 +00:00
Niels de Vos
0f9087d05e cephfs: connect snapshot to Ceph cluster in newSnapshotOptionsFromID()
Without connection, follow-up oparations on the volumeOptions object
will cause a panic. This should fix a regression in CephFS testing.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-19 09:15:43 +00:00
Niels de Vos
157d2aff64 cephfs: implement getFsName() with go-ceph
Fixes: 
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-15 13:54:57 +00:00
Niels de Vos
0a3f9d3487 cephfs: make getFsName() a method of volumeOptions
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-15 13:54:57 +00:00
Niels de Vos
f646840779 cephfs: implement getFscID() with go-ceph
Fixes: 
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-15 12:33:34 +00:00
Niels de Vos
b28ff63a29 cephfs: make getFscID() a method of volumeOptions
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-15 12:33:34 +00:00
Niels de Vos
5598c279da cleanup: no magic number for RWX permissions
golang-ci complains that 0777 is a magic number, so make it a constant
with a clear description.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-14 15:16:56 +00:00
Niels de Vos
9bd10d2698 cephfs: newVolumeOptions() does not need all secrets, only credentials
There is no need to pass all secrets on to newVolumeOptions(), it only
needs the credentials. As the caller of newVolumeOptions() already has
the credentials generated, just pass them along instead of the raw
secrets.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-14 15:16:56 +00:00
Niels de Vos
66046e4312 cephfs: drop credentials from createVolume()
The credentials are not used anymore, the volume object is already
connected to the cluster when createVolume() is called.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-14 15:16:56 +00:00
Niels de Vos
8cba08189d cephfs: implement createVolume() with go-ceph
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-14 15:16:56 +00:00
Niels de Vos
0f108edc7b cephfs: connect to the Ceph Cluster with go-ceph
Add the ClusterConnection to the volumeOptions type, so that future use
of go-ceph can connect to the Ceph cluster.

Once a volumeOptions object is not needed anymore, it needs to get
destroyed to free associated resources like the ClusterConnection.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-14 15:16:56 +00:00
Niels de Vos
f6de345cc9 cleanup: standardize error logs in cephfs/volume.go
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-01 12:00:59 +00:00
Humble Chirammal
b5d1edd3f7 cephfs: correct cephFSCloneInprogress source code comment
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-09-25 17:36:14 +00:00
Madhu Rajanna
89b326c896 cephfs: check ENOTEMPTY when removing subvolume
return a proper error message to the user when
the subvolume has the snapshots and it cannot
be removed until the snapshots on the subvolume
have to be deleted.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-09-23 13:51:23 +00:00
Niels de Vos
6f2f972f5b cephfs: trim spaces around fuseMountOptions
When passing

    fuseMountOptions: debug

in the StorageClass, the mount options passed on the ceph-fuse
commandline result in "-o nonempty ,debug". The additional space before
the ",debug" causes the mount command to fail.

Fixes: 1485
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-09-23 12:12:51 +00:00
Yug
0682f0e500 cephfs: Fetch subvolume features in newSnapshotOptionsFromID
Get subvolume features using subvolume info and update the
volume options with Features parameter.

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-09-16 16:05:28 +00:00
Yug
9e959c45a3 cephfs: No-Op Unprotect call if "snapshot-autoprotect" feature present
The subvolume features consists the list of features, which if includes
"snapshot-autoprotect", result in query based approach to
detecting the need for protect/unprotect.
If "snapshot-autoprotect" feature is present, The UnprotectSnapshot
call should be treated as a no-op

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-09-16 16:05:28 +00:00
Yug
43a4c100ec cephfs: No-Op Protect call if "snapshot-autoprotect" feature present
The subvolume features consists the list of features, which if includes
"snapshot-autoprotect", result in query based approach to
detecting the need for protect/unprotect.
If "snapshot-autoprotect" feature is present, The ProtectSnapshot
call should be treated as a no-op

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-09-16 16:05:28 +00:00
Yug
365eb58ff7 cephfs: use subvolume info if available
Use subvolume info to fetch the subvolume path.
If `subvolume info` command is not available,
use `getpath` command instead.

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-09-10 15:03:45 +00:00
Yug
59fc4aa00f cephfs: validate if subvolume has snapshot-retention feature
Add validateSnapshotRetention() to verify if subvolume has
`snapshot-retention` feature.

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-09-10 15:03:45 +00:00
Yug
6351358752 cephfs: Add --retain-snapshots flag to purge volume operations
Snapshots can be retained even after subvolume deletion in
Ceph 14.2.12. Adding support for the same in ceph-csi.

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-09-10 15:03:45 +00:00
Madhu Rajanna
860b65c446 cephfs: allow readonly only for volume with contentsource
It doesnot make sense to allow the creation of empty
volumes with readonly access, this commit allows the
creation of volume which is having readonly capabilities
only if the content source is set for the volume.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-09-10 13:47:23 +00:00
Madhu Rajanna
0bd45a5eb1 cephfs: use string const for cli error comparison
we should not return the CLI errors in GRPC errors
we need to return proper readable error messages
to the user for better understanding and better
debugging.

updates 

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-19 17:09:12 +00:00
Madhu Rajanna
787ba69089 cephfs: fix lock release in volumeDelete
currently the lock is not released which is
taken on the request name. this is causing issues
when the subvolume is requested for delete.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-18 06:12:43 +00:00
Madhu Rajanna
31c7c95fdb cephfs: check error output contains Error: ENOENT
execCommandErr returns both error and stderror
message. checking strings.HasPrefix is not helpful
as the stderr will be the first string. its good
to do string comparison and find out that error
is volume not found error.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-18 06:12:43 +00:00
Madhu Rajanna
f566b0fc2d cephfs: replace Warningf with WarningLog in unmountVolume
replace Errorf with ErrorLog in unmountVolume
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
3fe3d25dda cephfs: replace Errorf with ErrorLogMsg in loadAvailableMounters
replace Errorf with ErrorLog in loadAvailableMounters
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
fbe090065d cephfs: replace Errorf with ErrorLog in purgeVolume
replace Errorf with ErrorLog in purgeVolume
for context based logging

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
5a1652e5e8 cephfs: replace Errorf with ErrorLog in resizeVolume
replace Errorf with ErrorLog in resizeVolume
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
a882c1194b cephfs: replace Errorf with ErrorLog in createVolume
replace Errorf with ErrorLog in createVolume
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
f3d9b8933b cephfs: replace Errorf with ErrorLog in getSubVolumeInfo
replace Errorf with ErrorLog in getSubVolumeInfo
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
2b7aade289 cephfs: replace Errorf with ErrorLog in getVolumeRootPathCeph
replace Errorf with ErrorLog in getVolumeRootPathCeph
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
d1ac832266 cephfs: replace Errorf with ErrorLog in genSnapFromOptions
replace Errorf with ErrorLog in genSnapFromOptions
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
e83a51c1e1 cephfs: replace Errorf with ErrorLog in parseTime
replace Errorf with ErrorLog in parseTime
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
e7a043271d cephfs: replace Errorf with ErrorLog in cloneSnapshot
replace Errorf with ErrorLog in cloneSnapshot
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
841a3713f5 cephfs: replace Errorf with ErrorLog in unprotectSnapshot
replace Errorf with ErrorLog in unprotectSnapshot
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
91811c18c5 cephfs: replace Errorf with ErrorLog in protectSnapshot
replace Errorf with ErrorLog in protectSnapshot
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
72dfe6a265 cephfs: replace Errorf with ErrorLog in getSnapshotInfo
replace Errorf with ErrorLog in getSnapshotInfo
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
7e8cc41aca cephfs: replace Errorf with ErrorLog in deleteSnapshot
replace Errorf with ErrorLog in deleteSnapshot
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
7f00fa66f4 cephfs: replace Errorf with ErrorLog in createSnapshot
replace Errorf with ErrorLog in createSnapshot
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
57454cf379 cephfs: replace Errorf with ErrorLog in nodeUnstageVolume
replace Errorf with ErrorLog in nodeUnstageVolume
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
0a5581b7a0 cephfs: replace Errorf with ErrorLog in NodeUnPublishVolume
cephfs: replace Errorf with ErrorLog in NodeUnPublishVolume
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
79dbc41d14 cephfs: replace Errorf with ErrorLog in NodePublishVolume
replace Errorf with ErrorLog in NodePublishVolume
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
48a313c6cc cephfs: replace Errorf with ErrorLog in mount
replace Errorf with ErrorLog in mount for
context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
032901b8d0 cephfs: replace Errorf with ErrorLog in nodeStageVolume
replace Errorf with ErrorLog in nodeStageVolume
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00