Commit Graph

351 Commits

Author SHA1 Message Date
Niels de Vos
4005585806 util: store EnvVaultInsecure as string, not bool
The configuration option `EnvVaultInsecure` is expected to be a string,
not a boolean. By converting the bool back to a string (after
verification), it is now possible to skip the certificate validation
check by setting `vaultCAVerify: false` in the Vault configuration.

Fixes: #1852
Reported-by: Bryon Nevis <bryon.nevis@intel.com>
Signed-off-by: Niels de Vos <ndevos@redhat.com>
(cherry picked from commit 582d004ca5)
2021-02-02 12:43:48 +00:00
Mudit Agarwal
08904884d3 cephfs: ignore BytesQuota field in case it is not set.
This can happen when the subvolume is in snapshot-retained state.
We should not return error for such case as it is a valid situation.

Signed-off-by: Mudit Agarwal <muagarwa@redhat.com>
(cherry picked from commit d480eb4bda)
2021-02-01 10:35:39 +00:00
Ilya Dryomov
64f3361ff4 rbd: enable mapping and unmapping from a network namespace
Make rbdplugin pod work in a non-initial network namespace (i.e. with
"hostNetwork: false") by skipping waiting for udev events when mapping
and unmapping images.  CSI use case is very simple: all that is needed
is a device node which is immediately fed to mkfs, so we should be able
to tolerate udev not being finished with the device just fine.

Fixes: #1323
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 04644c1d58)
2021-01-13 17:28:25 +00:00
Ilya Dryomov
ca59d53a60 rbd: introduce appendDeviceTypeAndOptions()
Factor out --device-type and --options formatting.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit c2493686b7)
2021-01-13 17:28:25 +00:00
Ilya Dryomov
6918cbe950 rbd: rename ndbType parameter
Fix "ndb" typo.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit d3f31187fc)
2021-01-13 17:28:25 +00:00
Ilya Dryomov
2a090b785c rbd: rename mapOptions and options argument slices
With the new support for passing --options, referring to ExecCommand()
argument slices as mapOptions and options is confusing.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
(cherry picked from commit 5631b83dd0)
2021-01-13 17:28:25 +00:00
Seena Fallah
cb3ba5f58f rbd: fix namespace json parser for xbdDeviceInfo
rbd device list --format=json returns namespace as a namespace not radosNamespace

Signed-off-by: Seena Fallah <seenafallah@gmail.com>
(cherry picked from commit fdec9f65b8)
2021-01-06 10:40:16 +00:00
Madhu Rajanna
43fde0a30a cleanup: add a helper function storeImageID
added a helper function storeImageID to reduce
code duplication.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-07 11:03:27 +00:00
Madhu Rajanna
b2fb43b335 cleanup: reduce the code complexity of controller
created a new helper function to getCredentials.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-07 11:03:27 +00:00
Madhu Rajanna
e243c0006b rbd: dont generate OMAP data for static volume
if the user has created a static PV for a RBD
image which is not created by CSI driver, dont
generate the OMAP data.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-07 11:03:27 +00:00
Madhu Rajanna
c40872df00 rbd: undo reservation incase of errors
If cephcsi encounters any error after
reservation, as a cleanup operation
it should revert back the reservation.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-07 11:03:27 +00:00
Madhu Rajanna
99dbe27921 rbd: return nil if the omap data exists
If the omap data already exits return nil.
so that omap generator will not try to reserve
anything again.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-07 11:03:27 +00:00
Madhu Rajanna
ebb413534f cephfs: remove unsed cr util.Credentials
remove unused cr util.Credentials variable.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-03 14:53:33 +00:00
Madhu Rajanna
0f451ed465 cephfs: implement getSnapshotInfo with go-ceph
implement getSnapshotInfo function with go-ceph
to get subvolume information.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-03 14:53:33 +00:00
Madhu Rajanna
ddf91de859 cephfs: implement clonesnapshot with go-ceph
updated ceph fs CLI implementaion of cloning
with go-ceph.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-03 06:39:18 +00:00
Madhu Rajanna
814bf4459a cephfs: implement snapshot protect and unprotect to go-ceph
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-03 06:39:18 +00:00
Niels de Vos
3433cefaa5 cleanup: standardize logging in deleteSnapshot
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-02 14:36:39 +00:00
Niels de Vos
294f7b22d4 cephfs: remove unused Credentials argument from deleteSnapshot()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-02 14:36:39 +00:00
Niels de Vos
9d9b5b3303 cephfs: implement deleteSnapshot() with go-ceph
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-02 14:36:39 +00:00
Niels de Vos
032db78d8a cephfs: make deleteSnapshot() a method of volumeOptions
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-02 14:36:39 +00:00
Niels de Vos
5883f244d2 cleanup: standardize logging in createSnapshot
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-02 13:18:47 +05:30
Niels de Vos
7c580af412 cephfs: remove unused Credentials argument from createSnapshot()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-02 13:18:47 +05:30
Niels de Vos
1d8cc75282 cephfs: implement createSnapshot() with go-ceph
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-02 13:18:47 +05:30
Niels de Vos
57f8ee0aa7 cephfs: make createSnapshot() a method of volumeOptions
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-02 13:18:47 +05:30
Madhu Rajanna
a35a835e9c cephfs: fix logic to call ceph fs resize
clusterAdditionalInfo map is holding a localClusterState
for checking ceph cluster supports resize and subvolumegroup
is created or not, currently we are checking if the key is present
in a map and localClusterStatelocalClusterState.resizeSupported
is set to false to call ceph fs subvolume resize to check command is
supported or not, if a structure is  initialized all its members
are set to default value. so we will never going to check the
ceph fs subvolume  resize command is supported in backend or not, we are
always using ceph fs subvolume create to resize subvolume. in some
ceph version ceph fs subvolume create wont work to resize a subvolume.
This commit changes the resizeSupported from bool to *bool for
proper handling of this scenario.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-01 13:31:39 +00:00
Madhu Rajanna
a1c8942e2b cephfs: remove unused invalidCommand error const
removed unused invalidCommand from cephfs
error const.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-01 09:46:52 +00:00
Madhu Rajanna
1ba160afdb cephfs: handle invalid error in subvolume info
go ceph returns NotImplementedError for invalid
commands,cephcsi is using errors.As to find out
the error.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-01 09:46:52 +00:00
Madhu Rajanna
66eb3b7525 cephfs: handle invalid error in fs resize
go ceph returns NotImplementedError for invalid
commands,cephcsi is using errors.As to find out
the error.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-01 09:46:52 +00:00
Madhu Rajanna
2232067e99 cephfs: use rados.ErrNotFound in subvolume info
go-ceph rados.ErrNotFound for not found errors,cephcsi
need to check same for not found errors.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-01 09:46:52 +00:00
Madhu Rajanna
a0283ef7f9 cephfs: use rados.ErrNotFound in getVolumeRootPathCeph
go-ceph rados.ErrNotFound for not found errors,cephcsi
need to check same for not found errors.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-01 09:46:52 +00:00
Madhu Rajanna
b40d561228 cephfs: make getVolumeRootPathCeph as a method
getVolumeRootPathCeph is a method of volumeOptions

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-01 09:46:52 +00:00
Madhu Rajanna
e4713bc9ed cephfs: remove unused Credentials argument from getVolumeRootPathCeph
remove unused Credentials argument from getVolumeRootPathCeph
function.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-01 09:46:52 +00:00
Madhu Rajanna
b6f3ba885e cephfs: implement getVolumeRootPathCeph with go-ceph
instead of ceph fs CLI commands using go-ceph library
to get subvolume path.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-01 09:46:52 +00:00
Niels de Vos
5565a902ac rebase: split vaultConnection from VaultKMS
In order to re-use the configuration of Vault, split a new
vaultConnection type from the VaultKMS type.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-30 19:16:39 +00:00
Madhu Rajanna
8ebb9a1ba0 cleanup: fix misspell words
fixed misspell words detected by  codespell

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-11-30 08:46:48 +01:00
Madhu Rajanna
39b1f2b4d3 cleanup: fix mispell words
fixed mispell words in the repo.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-11-29 12:47:46 +05:30
Niels de Vos
8d08e8b37e cephfs: remove unused Credentials argument from getCloneState()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-29 07:13:35 +00:00
Niels de Vos
055975f790 cephfs: implement getCloneState() with go-ceph
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-29 07:13:35 +00:00
Niels de Vos
476ad18d2e cephfs: make getCloneState() a method of volumeOptions
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-29 07:13:35 +00:00
Niels de Vos
66aa595e77 cephfs: do not export internal CloneStatus type
getCloneInfo() does not need to return a full CloneStatus struct that
only has one member. Instead, it can just return the value of the single
member, so the JSON type/struct does not need to be exposed.

This makes the API for getCloneInfo() a little simpler, so it can be
replaced by a go-ceph implementation later on.

As the function does not return any of the unused attributes anymore, it
is renamed to getCloneStatu() as well.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-29 07:13:35 +00:00
Niels de Vos
7824cb5ed7 util: use %w when forwarding an error
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-29 04:03:59 +00:00
Niels de Vos
5fba89f783 cleanup: use libopenstorage/secrets for Vault access
Instead of the hand-rolled Vault usage, use the libopenstorage/secrets
package that provides a nice API. The support for Vault becomes much
simpler and maintainable that way.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-29 04:03:59 +00:00
Madhu Rajanna
6091490393 rbd: improve logging in getCloneDepth
earlier if the depth check fails the
complete vol struct was getting logged,
this commits logs only the pool and image
name.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-11-28 18:50:00 +00:00
Madhu Rajanna
b3120926b9 rbd: remove extra Destory of parent volume
removed extra Destory of the parent volume.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-11-28 18:50:00 +00:00
Madhu Rajanna
30af703a2f rbd: add controller to main
initialize and start the rbd controller when
we the driver type is controller.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-11-28 18:50:00 +00:00
Madhu Rajanna
68bd44beba rbd: add new controller to regenerate omap data
In the case of Disaster Recovery failover, the
user expected to create the static PVC's. We have
planned not to go with the PVC name and namespace
for many reasons (as in kubernetes it's planned to
support PVC transfer to a new namespace with a
different name and with new features coming in
like data populator etc). For now, we are
planning to go with static PVC's to support
async mirroring.

During Async mirroring only the RBD images are
mirrored to the secondary site, and when the
user creates the static PVC's on the failover
we need to regenerate the omap data. The
volumeHandler in PV spec is an encoded string
which contains clusterID and poolID and image UUID,
The clusterID and poolID won't remain same on both
the clusters, for that cephcsi need to generate the
new volume handler and its to create a mapping
between new volume handler and old volume handler
with that whenever cephcsi gets csi requests it
check if the mapping exists it will pull the new
volume handler and continues other operations.

The new controller watches for the PVs created,
It checks if the omap exists if it doesn't it
will regenerate the entire omap data.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-11-28 18:50:00 +00:00
Madhu Rajanna
14700b89d1 rbd: update inuse logic of a rbd image
in case of mirrored image, if the image is
primary a watcher will be added by the rbd
mirror deamon on the rbd image.
we have to consider 2 watcher to check image
is in use.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-11-28 18:50:00 +00:00
Madhu Rajanna
ba84f14241 journal: create object with provided UUID
incase of async mirroring the volume UUID is
retrieved from the volume name, instead of cephcsi
generating a new UUID it should reserve the passed
UUID it will be useful when we support both metro DR
and async mirroring on a kubernetes clusters.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-11-28 18:50:00 +00:00
Madhu Rajanna
8d3a44d0c4 rbd: add minsnapshotsonimage flag
An rbd image can have a maximum number of
snapshots defined by maxsnapshotsonimage
On the limit is reached the cephcsi will
start flattening the older snapshots and
returns the ABORT error message, The Request
comes after this as to wait till all the
images are flattened (this will increase the
PVC creation time.  Instead of waiting till
the maximum snapshots on an RBD image, we can
have a soft limit, once the limit reached
cephcsi will start flattening the task to
break the chain. With this PVC  creation time
will only be affected when the hard limit
(minsnapshotsonimage) reached.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-11-18 05:59:20 +00:00
Yug
3ac6bbd87c cephfs: Add isCloneRetryError function
The function isCloneRetryError verifies
if the clone error is `pending` or
`in-progress` error.

Co-authored-by: Madhu Rajanna <madhupr007@gmail.com>
Signed-off-by: Yug <yuggupta27@gmail.com>
2020-11-09 07:29:12 +00:00