1
0
mirror of https://github.com/ceph/ceph-csi.git synced 2024-12-27 15:30:23 +00:00
Commit Graph

1157 Commits

Author SHA1 Message Date
Humble Chirammal
38d005e4e6 cephfs: validate create volume request
Also add functionality to generate snap from request
and to get mon and clusterID from the request

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-08 00:11:35 +00:00
Humble Chirammal
9c000add29 cephfs: Change checkVolExist for snapshot and clone workflow
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-08 00:11:35 +00:00
Humble Chirammal
c773097f85 cephfs: add snapshot and clone helper functions
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-08 00:11:35 +00:00
Humble Chirammal
d1fe12b4f0 cephfs: change createvolume for snapshot/clone workflow
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-08 00:11:35 +00:00
Humble Chirammal
c31f349c6f cephfs: use delete operation lock in delete volume
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-08 00:11:35 +00:00
Humble Chirammal
f9257ed1cb cephfs: add snapshot create/delete capabilities in controllerserver
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-08 00:11:35 +00:00
Humble Chirammal
5a71949dcd cephfs: introduce newSnapshotOptionsFromID to generate volOpt and sid
From provided CSI volume ID this populate volumeOptions and snapshot
identifier after connecting to the snapJournal.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-08 00:11:35 +00:00
Humble Chirammal
a4f2c5617c cephfs: retrieve NewCSISnapshotJournal with namespace set
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-08 00:11:35 +00:00
Humble Chirammal
1746b4e6ee cephfs: add various error strings for command validation
Compared to previous version of the error strings, this change
depend on error strings like ENOENT, EEXIST, EINVAL..etc
The format of the error strings change in different cluster versions
and the error code return should not change. This also add extra
error strings for snapshot and clone operation outputs

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-08 00:11:35 +00:00
Humble Chirammal
6dcf03a4bc cephfs: add checkContentSource() to validate the data source
checkContentSource() validate the data source in the request
and then populate volumeOptions or snapshotshot identifier in
case of snapshot source. If the data source is volume, then
parentVolumeOption and pvID are populated.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-08 00:11:35 +00:00
Humble Chirammal
2bb0bbde7f cephfs: change createBackingVolume to accomodate clone operations
adjust createBackingVolume() to create a subvolume from snapshot
or existing subvolume by taking restore or clone operation locks

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-08 00:11:35 +00:00
Humble Chirammal
c968380134 cephfs: add snapshotlock and operationlock to controllerserver
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-08 00:11:35 +00:00
Humble Chirammal
66827facc3 cephfs: make use of expand operation lock in ControllerExpandVolume
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-08 00:11:35 +00:00
Madhu Rajanna
2458ec6573 rbd: return error if fetching cluster id fails
if we are not able to fetch the cluster-ID from
the createSnapshot request and also if we are
not able to get the monitor information from
the cluster-ID return error instead of using
the parent image information.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-03 14:25:06 +00:00
Mudit Agarwal
9ed0811422 rbd: implement rbdVolume.resize() with go-ceph
Replaced command execution with go-ceph Resize() function.
Volsize is being updated before waiting for resize() to return,
fixed it to get updated only after resize() is successful.

Signed-off-by: Mudit Agarwal <muagarwa@redhat.com>
2020-08-03 10:50:01 +00:00
Niels de Vos
be9e7cf956 rbd: pass context.Context to rbdVolume.resize()
While adding the context.Context to the resizeRBDimage() function, it
became a little ugly. So renaming the function to resize() and making it
a method of the rbdVolume type.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-24 16:04:13 +00:00
Niels de Vos
36469b87e2 util: make ExecComand return stdout and stderr as string
Most consumers of util.ExecCommand() need to convert the returned []byte
format of stdout and/or stderr to string. By having util.ExecCommand()
return strings instead, the code gets a little simpler.

A few commands return JSON that needs to be parsed. These commands will
be replaced by go-ceph implementations later on. For now, convert the
strings back to []byte when needed.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-24 16:04:13 +00:00
Niels de Vos
ddac66d76b util: use context.Context for logging in ExecCommand
All calls to util.ExecCommand() now pass the context.Context. In some
cases this is not possible or needed, and util.ExecCommand() will not
log the command.

This should make debugging easier when command executions fail.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-24 16:04:13 +00:00
Niels de Vos
bb4f1c7c9d rbd: use util.ExecCommand() instead of execCommand()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-24 16:04:13 +00:00
Niels de Vos
457d846241 cephfs: use util.ExecCommand() instead of execCommand()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-24 16:04:13 +00:00
Niels de Vos
47d5b60af8 rbd: disable reflink while creating XFS filesystems
Current versions of the mkfs.xfs binary enable reflink support by
default. This causes problems on systems where the kernel does not
support this feature. When the kernel the feature does not support, but
the filesystem has it enabled, the following error is logged in `dmesg`:

    XFS: Superblock has unknown read-only compatible features (0x4) enabled

Introduce a check to see if mkfs.xfs supports the `-m reflink=` option.
In case it does, pass `-m reflink=0` while creating the filesystem.

The check is executed once during the first XFS filesystem creation. The
result of the check is cached until the nodeserver restarts.

Fixes: 
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-24 13:37:51 +00:00
Niels de Vos
526da43b6a rbd: remove unused rbdStatus()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-24 11:34:48 +00:00
Niels de Vos
7afaac9c66 rbd: implement rbdVolume.isInUse() with go-ceph
The new rbdVolume.isInUse() method will replace the rbdStatus()
function. This removes one more rbd command execution in the
DeleteVolume path.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-24 11:34:48 +00:00
Sven Anderson
92884f56f4 rbd: simplify error handling
This change replaces the sentinel errors in rbd module with
standard errors created with errors.New().

Related: 

Signed-off-by: Sven Anderson <sven@redhat.com>
2020-07-23 11:16:40 +00:00
Sven Anderson
dba2c27bcb cephfs: simplify error handling
This change replaces the sentinel errors in cephfs module with
standard errors created with errors.New().

Related: 

Signed-off-by: Sven Anderson <sven@redhat.com>
2020-07-23 11:16:40 +00:00
Sven Anderson
7c9c7c78a7 util: add tests for JoinErrors()
Signed-off-by: Sven Anderson <sven@redhat.com>
2020-07-23 11:16:40 +00:00
Sven Anderson
8393fbe40b util: simplify error handling
The sentinel error code had additional fields in the errors, that are
used nowhere.  This leads to unneccesarily complicated code.  This
change replaces the sentinel errors in utils with standard errors
created with errors.New() and adds a simple JoinErrors() function to
be able to combine sentinel errors from different code tiers.

Related: 

Signed-off-by: Sven Anderson <sven@redhat.com>
2020-07-23 11:16:40 +00:00
Madhu Rajanna
b3a4f510e6 rbd: take operation locks before operating on resource
Take operation locks on the resources before operating
on the resouces. This allows us to do parallel operations
for some RPC calls such as Clone and Restore of PVC.
This operations will only be blocked if the image is
expanding or Snapshot and RBD image is getting deleted.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-21 13:25:19 +00:00
Madhu Rajanna
d6348545ab journal: Add additional operation based locking
As we are adding new functionalities like Create/Delete
snapshot,Clone from Snapshot and Clone from Volume.
with the current implementation, there are only serial
operations allowed for this functionalities, for some
function we can allow parallel operations like
Clone from snapshot and Clone from Volume and Create
`N` snapshots on a single volume.

Delete Volume: Need to ensure that there is no clone,
Snapshot create and  Expand volume in progress.

Expand Volume: Need to ensure that there is no clone,
snapshot create and cloning in progress

Delete Snapshot: Need to ensure that there is no
cloning in progress

Restore Volume/Snapshot: Need to ensure that there is
no Expand or delete operation in progress.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-21 13:25:19 +00:00
Yug
71ddf51544 cleanup: address gomnd warnings
Direct usage of numbers should be avoided.

Issue reported:
mnd: Magic number: X, in <argument> detected (gomnd)

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-07-21 08:36:24 +00:00
Yug
e73fe64a0d cleanup: address gosec warnings
gosec warns about security problems by scanning the
Go AST.

Issues Reported:
G101 (CWE-798): Potential hardcoded credentials (Confidence: LOW, Severity: HIGH)
G204 (CWE-78): Subprocess launched with variable (Confidence: HIGH, Severity: MEDIUM)
G304 (CWE-22): Potential file inclusion via variable (Confidence: HIGH, Severity: MEDIUM)

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-07-21 08:36:24 +00:00
Yug
48fa43270f cleanup: address gocritic warnings
Add explanation to nolint directives.

Issue reported:
whyNoLint: include an explanation for nolint directive (gocritic)

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-07-21 08:36:24 +00:00
Yug
628ae9e982 cleanup: use wrapped static errors instead of dynamic
In Go 1.13, the fmt.Errorf function supports a new %w verb.
When this verb is present, the error returned by fmt.Errorf
will have an Unwrap method returning the argument of %w,
which must be an error. In all other ways, %w is identical to %v.

Updates: 

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-07-21 08:36:24 +00:00
Yug
7f94a57908 cleanup: address godot warnings
Top level comments should end in a period

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-07-21 08:36:24 +00:00
Madhu Rajanna
11a6f6c1dd rbd: Support data-pool when cloning rbd image
Added support to clone an image in data-pool
during CreateVolume RPC call.

updates 

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-20 17:29:48 +00:00
Madhu Rajanna
1e5370a1f3 cephfs: return volume not found error if volume doesnot exists
In some ceph version if the subvolume is not present, the
ceph returns doesnot exists and in some version not found
error message. This commit fixes issue for both error
checks.

By only checking Error ENOENT: for doesnot exist seems good.
even if some error message changes in ceph ceph-csi wont get
any issue.

```bash
sh-4.2# ceph version
ceph version 14.2.10 (b340acf629a010a74d90da5782a2c5fe0b54ac20) nautilus (stable)

sh-4.2# ceph fs subvolume getpath myfs csi-vol-a24a3d97-c7f4-11ea-8cfc-0242ac110012 --group_name csi
Error ENOENT: subvolume 'csi-vol-a24a3d97-c7f4-11ea-8cfc-0242ac110012' does not exist
```

```bash
sh-4.2# ceph version
ceph version 14.2.4 (75f4de193b3ea58512f204623e6c5a16e6c1e1ba) nautilus (stable)

sh-4.2# ceph fs subvolume getpath myfs testing --group_name=csi
Error ENOENT: Subvolume 'testing' not found
```

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-17 10:27:18 +00:00
Madhu Rajanna
684cb13c54 rbd: DisAllow CreateVoulume for missmatch volume size
If the requested volume size and the snapshot or the
parent volume from which the clone is to be created
is not equal cephcsi returns an error message.

updates 

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-16 12:33:27 +00:00
Madhu Rajanna
5208c0fc38 cleanup: replace klog with v2
This commit replaces the klog with klog/v2
in leftover place.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-16 04:10:58 +00:00
Madhu Rajanna
2c67ba1ec4 rbd: Return current depth if the image is not found
If the image in the chain is moved to trash, we
cannot get the image details. We need to return the
found depth to the caller.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-15 18:40:45 +00:00
Madhu Rajanna
76c2f3c109 cleanup: re-use flattenTemporaryClonedImages to reduce duplicate code
re-use flattenTemporaryClonedImages to avoid code duplication

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-15 16:10:38 +00:00
Madhu Rajanna
8fc9146056 rbd: flatten temp cloned images
If the snapshots on the parent image exceeds
maxSnapshotsOnImage count, we need to flatten
all the temporary cloned images to over come the
krbd issue of maximum number of snapshots on
an image.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-15 16:10:38 +00:00
Madhu Rajanna
2fe1ee5287 rbd: create temporary snapshot with name same as temporary clone
create temporary snapshot on the parent image same as
name as the temporary clone rbd image. Naming the snapshot
and the temporary cloned image helps to flatten the temporary
cloned images when the snapshots on the parent image exceeds
the configured maxSnapshotsOnImage.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-15 16:10:38 +00:00
Madhu Rajanna
09ffaee7c3 cleanup: rename newVolumeOptionsFromVersion1Context for more clarity
rename newVolumeOptionsFromVersion1Context to newVolumeOptionsFromMonitorList
to provide more clarity to the function readers and also fixed comments.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-13 04:04:28 +00:00
Madhu Rajanna
d15ded88f5 cleanup: Remove support for Delete and Unmounting v1.1.0 PVC
as v1.0.0 is deprecated we need to remove the support
for it in the Next coming (v3.0.0) release. This PR
removes the support for the same.

closes 

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-10 16:07:13 +00:00
Madhu Rajanna
a0fd805a8b rbd: Add support for smart cloning
Added support for RBD PVC to PVC cloning, below
commands are executed to create a PVC-PVC clone from
RBD side.

* Check the depth(n) of the cloned image if n>=(hard limit -2)
or ((soft limit-2) Add a task to flatten the image and return
about (to avoid image leak) **Note** will try to flatten the
temp clone image in the chain if available
* Reserve the key and values in omap (this will help us to
avoid the leak as it's not reserved earlier as we have returned
ABORT (the request may not come back))
* Create a snapshot of rbd image
* Clone the snapshot (temp clone)
* Delete the snapshot
* Snapshot the temp clone
* Clone the snapshot (final clone)
* Delete the snapshot

```bash
1) check the image depth of the parent image if flatten required
add a task to flatten image and return ABORT to avoid leak
(hardlimit-2 and softlimit-2 check will be done)
2) Reserve omap keys
2) rbd snap create <RBD image for src k8s volume>@<random snap name>
3) rbd clone --rbd-default-clone-format 2 --image-feature
layering,deep-flatten <RBD image for src k8s volume>@<random snap>
<RBD image for temporary snap image>
4) rbd snap rm <RBD image for src k8s volume>@<random snap name>
5) rbd snap create <cloned RBD image created in snapshot process>@<random snap name>
6) rbd clone --rbd-default-clone-format 2 --image-feature <k8s dst vol config>
 <RBD image for temporary snap image>@<random snap name> <RBD image for k8s dst vol>
7)rbd snap rm <RBD image for src k8s volume>@<random snap name>
```

* Delete temporary clone image created as part of clone(delete if present)
* Delete rbd image

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-10 14:02:12 +00:00
Yug
1490daed7e cleanup: Avoid usage of numbers
Add seperate functions to handle all
levels and types of logging.

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-07-10 07:41:23 +00:00
Yug
8dc4ab6b1b rebase: update k8s.io/klog to v2.3.0
Update klog version to v2.3.0

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-07-10 07:41:23 +00:00
Sven Anderson
13f291dfc6 cleanup: use errors.As() for error type checks
Replaces some remaining old-style error type checks with errors.As()

Signed-off-by: Sven Anderson <sven@redhat.com>
2020-07-09 09:58:33 +00:00
Madhu Rajanna
4e82384b43 rbd: check image not found error
during the checkSnapCloneExists we are checking
the image, if the image not found we are deleting
the snapshot on the parent image, This PR corrects
the comparasion. instead of snapshotNotFound we need
to check ImageNotFound error.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-08 17:09:59 +00:00
Madhu Rajanna
82080dcfa9 util: update unit testing for deep flatten
updated unit testing for the kernel check
for deep flatten feature for both supported
upstream kernel version (5.1.0+) and RHEL
8.2 backport

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-08 05:48:42 +00:00
Yug
34593c9af0 cleanup: Reduce cyclomatic complexity of topology_test
Reduce redundant code to make the test simpler.

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-07-07 06:08:55 +00:00
Madhu Rajanna
f05c8ed2a7 journal: remove unused rados CLI functions
Removed unused rados CLI functions, as rados
CLI is replaced with go-ceph bindings.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-06 15:03:44 +00:00
Madhu Rajanna
9bb79fbe7d journal: replace getStoredImageId with go-ceph
moved implementation of getStoredImageID to go-ceph
to get better performance.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-06 15:03:44 +00:00
Madhu Rajanna
341a3c9f59 journal: replace rados CLI with go-ceph to storeimage id
Moved the implementation of storeImageID from rados
CLI to go-ceph api.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-06 15:03:44 +00:00
Madhu Rajanna
8f758450d8 rbd: add RHEL 8.2 kernel to the list
as RHEL 8.2 supports the deep-flatten
feature, added it to the list to map
the rbd images on the node without flattening.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-06 12:20:00 +00:00
Madhu Rajanna
04c8c7fd4a rbd: correct upstream kernel version for deep-flatten
as v5.1.0 supports the deep-flatten feature,lowering
the required version to map rbd images which
are having deep-flatten feature

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-06 12:20:00 +00:00
Madhu Rajanna
032fb228dd rbd: take lock on parent image during snapshotcreate
we need to take lock on parent rbd image when
we are creating a snapshot from it, if the user
tries to delete/resize the rbd image when we are
taking snapshots,we may face issues. if the volume
lock is present on the rbd image, the user cannot
resize the rbd image nor delete the rbd image.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-06 10:08:31 +00:00
Madhu Rajanna
61a81d35e8 rbd: flatten cloned images to freeup snapshot
flatten cloned images to remove the link
with the snapshot as the parent snapshot can
be removed from the trash.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-06 10:08:31 +00:00
Madhu Rajanna
8ef7143e6c rbd: add maxsnapshotsonimage flag
Added maxsnapshotsonimage flag to flatten
the older rbd images on the chain to avoid
issue in krbd.The limit is in krbd since it
only allocate 1 4KiB page to handle all the
snapshot ids for an image.

The max limit is 510 as per
https://github.com/torvalds/linux/blob/
aaa2faab4ed8e5fe0111e04d6e168c028fe2987f/drivers/block/rbd.c#L98
in cephcsi we arekeeping the default to 450 to reserve 10%
to avoid issues.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-06 10:08:31 +00:00
Madhu Rajanna
c04b903eca rbd: add listsnapshots function
added listsnapshots function for an
rbd image to list all the snapshots
created from an rbd images, This will
list the snapshots which are in trash also.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-06 10:08:31 +00:00
Sven Anderson
de74c36d8c cleanup: address err113 warnings about direct error comparisons
Several places in the code compared errors directly with the go-ceph
sentinel errors.  This change uses the errors.Is() function of go
1.13 instead.  The err113 linter reported this issue as:

err113: do not compare errors directly, use errors.Is() instead

Signed-off-by: Sven Anderson <sven@redhat.com>
2020-07-06 08:57:46 +00:00
Niels de Vos
267c709194 cleanup: use errors.As() in rbd/controllerserver methods
See-also: https://github.com/golang/go/wiki/ErrorValueFAQ#how-should-i-change-my-error-handling-code-to-work-with-the-new-features
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-03 09:12:48 +00:00
Niels de Vos
91ade42a32 cleanup: use errors.As() in journal.Connection methods
See-also: https://github.com/golang/go/wiki/ErrorValueFAQ#how-should-i-change-my-error-handling-code-to-work-with-the-new-features
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-03 09:12:48 +00:00
Niels de Vos
ad04e0d8c4 cleanup: use errors.As() in cephfs.checkVolExists()
See-also: https://github.com/golang/go/wiki/ErrorValueFAQ#how-should-i-change-my-error-handling-code-to-work-with-the-new-features
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-03 09:12:48 +00:00
Niels de Vos
72c74f5f8a cleanup: use errors.As() in cephfs.NodeStageVolume()
See-also: https://github.com/golang/go/wiki/ErrorValueFAQ#how-should-i-change-my-error-handling-code-to-work-with-the-new-features
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-03 09:12:48 +00:00
Niels de Vos
a47cfbc6c8 cleanup: use errors.As() in rbdVolume.Exists() and rbd.checkSnapExists()
See-also: https://github.com/golang/go/wiki/ErrorValueFAQ#how-should-i-change-my-error-handling-code-to-work-with-the-new-features
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-03 09:12:48 +00:00
Niels de Vos
d4dad7c189 cleanup: use errors.As() in rbd.NodeUnstageVolume()
See-also: https://github.com/golang/go/wiki/ErrorValueFAQ#how-should-i-change-my-error-handling-code-to-work-with-the-new-features
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-03 09:12:48 +00:00
Niels de Vos
7126e92b4f cleanup: use errors.As() in rbd.rbdStatus()
See-also: https://github.com/golang/go/wiki/ErrorValueFAQ#how-should-i-change-my-error-handling-code-to-work-with-the-new-features
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-03 09:12:48 +00:00
Niels de Vos
f93730dee7 cleanup: use errors.As() to compare errors in cephfs/controllerserver
See-also: https://github.com/golang/go/wiki/ErrorValueFAQ#how-should-i-change-my-error-handling-code-to-work-with-the-new-features
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-03 09:12:48 +00:00
Niels de Vos
57d0989593 cleanup: use errors.As() to compare errors in util/cephcmds
See-also: https://github.com/golang/go/wiki/ErrorValueFAQ#how-should-i-change-my-error-handling-code-to-work-with-the-new-features
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-03 09:12:48 +00:00
Niels de Vos
92aae4834e cleanup: use standard Golang errors package
"github.com/pkg/errors" does not offer more functionlity than that we
need from the standard "errors" package. With Golang v1.13 errors can be
wrapped with `fmt.Errorf("... %w", err)`. `errors.Is()` and
`errors.As()` are available as well.

See-also: https://tip.golang.org/doc/go1.13#error_wrapping
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-03 09:12:48 +00:00
Niels de Vos
8effa0cd3e util: add Unwrap() to error types
See-also: https://github.com/golang/go/wiki/ErrorValueFAQ#i-have-a-type-that-implements-error-and-holds-a-nested-error-how-should-i-adapt-it-to-the-new-features
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-03 09:12:48 +00:00
Niels de Vos
6d6b5bab3c rbd: add Unwrap() to error types
See-also: https://github.com/golang/go/wiki/ErrorValueFAQ#i-have-a-type-that-implements-error-and-holds-a-nested-error-how-should-i-adapt-it-to-the-new-features
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-03 09:12:48 +00:00
Niels de Vos
fad0cf7f1c cephfs: add Unwrap() to error types
See-also: https://github.com/golang/go/wiki/ErrorValueFAQ#i-have-a-type-that-implements-error-and-holds-a-nested-error-how-should-i-adapt-it-to-the-new-features
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-03 09:12:48 +00:00
Niels de Vos
0bafa2be8c doc: add comments to rbd error methods
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-03 09:12:48 +00:00
Niels de Vos
393569482e doc: add comments to cephfs error methods
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-03 09:12:48 +00:00
Niels de Vos
ec1a009dee rbd: allow DeleteVolume() to be too complex according to gocyclo
By fixing the golangci-lint runs, this now gets reported as a problem.
Instead of addressing the compexity of the DeleteVolume() method here,
mark it as a TODO.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-02 14:24:02 +00:00
Madhu Rajanna
7a87db39e6 rbd: check rbd image status only duing DeleteVolume RPC
currently, various calls to deleteImage does not
need the rbdStatus check. That is only required
when calling from DeleteVolume. This PR optimizes the
rbd image deletion by removing the status check.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-02 11:43:25 +00:00
Madhu Rajanna
f711d15551 rbd: store imageID if image is present
Added a check to make sure that the image
ID is stored if the image is already found.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-01 08:21:47 +00:00
Madhu Rajanna
b085577a4f rbd: add skipForceFlatten flag
added skipForceFlatten flag to skip
the image deptha and skip image flattening.
This will be very useful if the kernel is
not listed in cephcsi which supports deep
flatten fauture.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-01 08:21:47 +00:00
Madhu Rajanna
e3a63029a3 rbd: Implement snapshot and clone from snapshot
This Adds a support for create,delete snapshot
and creating a new rbd image from the snapshot.

* Create a snapshot

* Create a temporary snapshot from the parent volume
* Clone a new image from a temporary snapshot with options
  --rbd-default-clone-format 2 --image-feature layering,deep-flatten
* Delete temporary snapshot created
* Create a new snapshot from cloned image
* Check the image chain depth, if the Softlimit is reached Add a
  task Flatten the cloned image and return success. if the depth
  is reached hard limit Add a task Flatten the cloned image and
  return snapshot status ready as false

```bash
1) rbd snap create <RBD image for src k8s volume>@<random snap name>
2) rbd clone --rbd-default-clone-format 2 --image-feature
   layering,deep-flatten <RBD image for src k8s volume>@<random snap>
   <RBD image for temporary snap image>
3) rbd snap rm <RBD image for src k8s volume>@<random snap name>
4) rbd snap rm <RBD image for temporary snap image>@<random snap name>
5) check the depth, if the depth is greater than configured hard
   limit add a task to flatten the cloned image return snapshot status
   ready as false if the depth is greater than soft limit add a task
   to flatten the image and return success
```

* Create a clone from snapshot

* Clone a new image from the snapshot with user-provided options
* Check the depth(n) of the cloned image if n>=(hard limit)
  Add task to flatten the image and return ABORT (to avoid image leak)

```bash
1) rbd clone --rbd-default-clone-format 2 --image-feature
   <k8s dst vol config> <RBD image for temporary snap image>@<random snap name>
    <RBD image for k8s dst vol>
2) check the depth, if the depth is greater than configured hard limit
    add a task to flatten the cloned image return ABORT error if the depth is
    greater than soft limit add a task to flatten the image and return success
```

* Delete snapshot or pvc

* Move the temporary cloned image to the trash
* Add task to remove the image from the trash

```bash
1) rbd trash mv <cloned image>
2) ceph rbd task trash remove <cloned image>
```

With earlier implementation to delete the image, we used to add
a task to remove the image with new changes this cannot be done
as the image may contain snapshots or linking.so we will be
doing below steps to delete an image(this will be
applicable for both normal image and cloned image)

* Move the rbd image to the trash
* Add task to remove the image from the trash

```bash
1) rbd trash mv <image>
2) ceph rbd task trash remove <image>
```

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-01 08:21:47 +00:00
Madhu Rajanna
d95b07e39b rbd: implement rbd clone with go-ceph
moved the implementation of clone
from CLI to go-ceph

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-01 08:21:47 +00:00
Madhu Rajanna
c30f91eead rbd: implement snapinfo with go-ceph
moved snapinfo function from CLI to go-ceph
for better performance

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-01 08:21:47 +00:00
Madhu Rajanna
b3f1499cd4 rbd: remove unused snapshot function
we dont need to call the snapshot CLI functions
to get snapshot details. as these details are not
requried with new snapshot design.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-01 08:21:47 +00:00
Madhu Rajanna
f08118a8e5 rbd: Implement create and delete snapshot with go-ceph
Implement create and delete snapshot with go-ceph

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-01 08:21:47 +00:00
Madhu Rajanna
6ba0cd58b9 rbd: add code to store/retrieve the image ID
cephcsi need to store and retrieve the rbd image ID
in the omap as we need the image ID to add a
task to remove the image from the Trash.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-01 08:21:47 +00:00
Madhu Rajanna
3dcef09676 rbd: Use rbd trash commands to delete image
as with snapshot and cloning implementation
the rbd images cannot be deleted with rbd
remove or add a task to delete the rbd image
as it might contains the snapshots and clones.
we need to make use of the rbd mv trask and
add a task to remove the image from trash
once all its clones and snapshots links
are broken and there will no longer any
dependency between parent and child images.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-01 08:21:47 +00:00
Madhu Rajanna
9edb3b8e72 rbd: Add getImageID to get id of an image
add a new function called getImageID to fetch
the image id of an image which need to be stored
and retrived for Delete operation.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-01 08:21:47 +00:00
Madhu Rajanna
9b518726ab rbd: add hardlimt and softlimit flag
added Hardlimit and Softlimit flags for cephcsi
arguments. When the Softlimit is reached cephcsi
will start a background task to flatten the rbd
image and return success and if the hardlimit
is reached it will start a background task
to flatten the rbd image and return ready
to use as false to make sure that the image
will not be used until it is flatten.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-01 08:21:47 +00:00
Madhu Rajanna
3e7fa93256 rbd: Remove protect and unprotect function
As we are using v2 cloning we dont need to
do protect the snapshot before cloning and
unprotect it before deleting.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-01 08:21:47 +00:00
Madhu Rajanna
567b2ab280 util: Rename KernelVersion to GetKernelVersion
Renames KernelVersion to GetKernelVersion for
more readibility.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-01 08:21:47 +00:00
Madhu Rajanna
12c3be9974 util: Move supported version check to util
as we need to reuse the same code for both cephfs
and rbd moving the supported version check function
to util package, for better readability renamed
the function.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-01 08:21:47 +00:00
Niels de Vos
1b89f86d51 util: handle Close() errors in pidlimit
A new version of gosec insists on handling errors returned by Close():

    [/go/src/github.com/ceph/ceph-csi/internal/util/pidlimit.go:44] - G307 (CWE-): Deferring unsafe method "*os.File" on type "Close" (Confidence: HIGH, Severity: MEDIUM)
      > defer cgroup.Close()

    [/go/src/github.com/ceph/ceph-csi/internal/util/pidlimit.go:78] - G307 (CWE-): Deferring unsafe method "*os.File" on type "Close" (Confidence: HIGH, Severity: MEDIUM)
      > defer f.Close()

    [/go/src/github.com/ceph/ceph-csi/internal/util/pidlimit.go:113] - G307 (CWE-): Deferring unsafe method "*os.File" on type "Close" (Confidence: HIGH, Severity: MEDIUM)
      > defer f.Close()

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-01 05:56:32 +00:00
Niels de Vos
30dc83c3bb util: create ceph.conf with 0600 permissions
The generated ceph.conf does not need readable by the group, there is
only one (system) user consuming the configurations file.

This addresses the following gosec warning:

    [/go/src/github.com/ceph/ceph-csi/internal/util/cephconf.go:52] - G306 (CWE-): Expect WriteFile permissions to be 0600 or less (Confidence: HIGH, Severity: MEDIUM)
      > ioutil.WriteFile(CephConfigPath, cephConfig, 0640)

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-01 05:56:32 +00:00
Niels de Vos
16c4e33b09 util: log error when closing temp fail fails in GetOMapValue()
gosec-2.3.0 complains about the following:

    [/go/src/github.com/ceph/ceph-csi/internal/util/cephcmds.go:146] - G307 (CWE-): Deferring unsafe method "*os.File" on type "Close" (Confidence: HIGH, Severity: MEDIUM)
      > defer tmpFile.Close()

By logging the error from Close(), the warning is gone.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-01 05:56:32 +00:00
Madhu Rajanna
e1fd12fb29 cephfs: do chmod on stagingpath
instead of doing chmod on the targetpath always
do it once at the stagintpath.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-06-30 17:43:41 +00:00
Madhu Rajanna
4d11490a2b cephfs: Add support for ROX PVC
cephcsi need to add mount the cephfs subvolume
as the readonly when the PVC type is ROX to
provide only readonly access to the users

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-06-30 17:43:41 +00:00
Yug Gupta
c9ad904331 cephfs: Add support for multiple subvolumegroups
With the current code base, the subvolumegroup will
be created once, and even for a different cluster,
subvolumegroup creation is not allowed again.

Added support multiple subvolumegroups creation by
validating one subvolumegroup creation per cluster.

Fixes: 

Signed-off-by: Yug Gupta <ygupta@redhat.com>
2020-06-26 09:19:24 +00:00
Niels de Vos
efd9b31043 cleanup: use util.MountOptionsAdd() in CephFS volumemounter
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-06-24 13:33:44 +00:00
Niels de Vos
cdbc56de7d ci: add test for MountOptionsAdd()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-06-24 13:33:44 +00:00
Niels de Vos
887aab1d72 util: add MountOptionsAdd() to add mount options
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-06-24 13:33:44 +00:00
John Mulligan
75088aa36d util: add doc comments for exported functions in errors.go
A number of exported functions in errors.go were missing doc comments.
Add them.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-06-22 15:41:00 +00:00
John Mulligan
09e1c856d0 journal: do not return errors from remove omap func if omap missing
The previous function used to remove omap keys apparently did not
return errors when removing omap keys from a missing omap (oid).
Mimic that behavior when using the api.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-06-22 15:41:00 +00:00
John Mulligan
c14cbee132 journal: change omap set func to handle multiple key-value pairs
For any function that sets more than one key on a single oid setting
them as a batch will be more efficient.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-06-22 15:41:00 +00:00
John Mulligan
a8409eccc8 journal: change omap remove func to handle multiple keys at once
For any function that removes more than one key on a single oid removing
them as a batch will be more efficient.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-06-22 15:41:00 +00:00
John Mulligan
0ac5f40d09 journal: change omap get func to handle multiple keys at once
Taking this appraoch means that any function that must get more than one
key's value from the same oid can be more efficient by calling out to
ceph only once.

To be cautious and avoid missing things we always request ceph return
more keys than we actually expect to be set on the oid. If there are
unexpected keys there, we will not miss the keys we want if we first hit
an unexpected key if we were to limit ourselves to iterating only over
the number of keys we're expecting to be on the object.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-06-22 15:41:00 +00:00
John Mulligan
cd24bb3f5c journal: convert journal to use new omap functions
Convert the business-logic of the journal to use the new go-ceph based
omap manipulation functions.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-06-22 15:41:00 +00:00
John Mulligan
0b99bdaa82 journal: add omap.go with go-ceph based omap manipulation funcs
These new omap manipulation functions (get/set/remove) are roughly
equivalent to the previous command-line based approach but rely
on direct api calls to ceph.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-06-22 15:41:00 +00:00
John Mulligan
f3192bd1b3 util: add New* funcs for ErrKeyNotFound & ErrPoolNotFound types
These types have private fields but we need to construct them outside of
the util package. Add New* methods for both.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-06-22 15:41:00 +00:00
Madhu Rajanna
649aeb7aaf rbd: Add support for rbd ROX PVC mounting
if the PVC access mode is ReadOnlyMany
or single node readonly, mounting the rbd
device path to the staging path as readonly
to avoid the write operation.

If the PVC acccess mode is readonly, mapping
rbd images as readonly.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-06-22 06:15:40 +00:00
Niels de Vos
da40d8e05e rbd: use librbd.FeatureSet for features
go-ceph v0.3 adds constants for ImageFeature values and their names.
Instead of hardcoding "layering" in several places, use the constant
given by librbd.

The rbdVolume.ImageFeatures does not seem to be used anywhere after the
conversion. Stashing the image metadata does include the ImageFeatures
as these are retrieved when getting the image information. It is safe to
drop ImageFeatures altogether and only use the imageFeatureSet instead.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-06-19 11:37:29 +00:00
Niels de Vos
decfc1ae2c rbd: make hasSnapshotFeature() a method of rbdVolume
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-06-19 11:37:29 +00:00
Niels de Vos
ebe378c79a util: trim "\x00" characters from the kernel version
It seems that convering the release component from the unix.Utsrelease
type leaves some trailing "\x00" characters.

While splitting the string to compare kernel versions, these additional
characters might prevent converting the string to an int. Strip the
additional characters before returning the string.

Note:
  "\x00" characters are not visible when printing to a file or screen.
  They can be seen in hex-editors, or sending the output through 'xxd'.

Fixes: 
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-06-18 11:42:47 +00:00
Niels de Vos
209a5e5602 util: add function to get the kernel version
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-06-18 11:42:47 +00:00
Niels de Vos
3364fe7b78 rbd: remove unneeded updateVolWithImageInfo()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-06-17 09:28:57 +00:00
Niels de Vos
824f38daaa rbd: use go-ceph to implement getImageInfo()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-06-17 09:28:57 +00:00
Humble Chirammal
323cc0e3bb cleanup: avoid comparing errors directly
Go 1.13 contains support for error wrapping. To support wrapping,
fmt.Errorf now has a %w verb for creating wrapped errors, and three
new functions in the errors package ( errors.Unwrap, errors.Is and
errors.As) simplify unwrapping and inspecting wrapped errors.

With this change, If we currently compare errors using ==, we have to
use errors.Is instead. Example:

if err == io.ErrUnexpectedEOF

becomes

if errors.Is(err, io.ErrUnexpectedEOF)

https://tip.golang.org/doc/go1.13#error_wrapping

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-06-17 06:26:11 +00:00
Yug Gupta
5fbc153e0f cephfs: fix IneffAssign warnings in resizeVolume()
IneffAssign warns about the two following statements:

Line 147: warning: ineffectual assignment to supported (ineffassign)
Line 148: warning: ineffectual assignment to ok (ineffassign)

Reported-by: https://goreportcard.com/report/github.com/ceph/ceph-csi
Updates: 

Signed-off-by: Yug Gupta <ygupta@redhat.com>
2020-06-16 09:06:18 +00:00
Humble Chirammal
d11266e255 cleanup: remove double import of csi spec from nodeserver
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-06-11 10:28:24 +00:00
Niels de Vos
be703d1e42 util: implement CreateObject() with go-ceph
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-06-10 07:20:25 +00:00
Niels de Vos
6e24b10364 util: implement RemoveObject() with go-ceph
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-06-10 07:20:25 +00:00
Niels de Vos
926f1e813c rbd: keep rados.IOContext for re-use
This prevents the need to open the IOContext for additional operations
on the image.

It also addresses a leak of the IOContext in case `rbdVolume.open()` was
called. The method only returned the `rbd.Image` without the possibility
to close the related IOContext.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-06-09 13:17:31 +00:00
Niels de Vos
365fbd1ca0 util: remove unused GetImageMeta() and SetImageMeta()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-06-03 06:15:46 +00:00
Madhu Rajanna
2f7e51076b rbd: unmap rbd image if the mounting fails
There is a bug in current code where the devicePath
is always empty and the rbd image unmap never
happens if nodeplugin fails to mount the rbd image
to the stagingpath.
This is a fix to unmap the rbd image if some issue
occurs after rbd image is mapped.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-06-01 11:27:25 +00:00
Mehdy Khoshnoody
c0361c47d6 rbd: Add new methods to generate spec strings
Refactor lots of string concatenation using the Stringer
implementation for each type.

Signed-off-by: Mehdy Khoshnoody <mehdy.khoshnoody@gmail.com>
2020-06-01 05:54:54 +00:00
Mehdy Khoshnoody
fe384bccfe rbd: Fix error checking mistake in CreateVolume
deleteErr should be checked if it's nil instead of err

Signed-off-by: Mehdy Khoshnoody <mehdy.khoshnoody@gmail.com>
2020-05-29 07:38:01 +00:00
Mudit Agarwal
9fe7d1612e journal: In mount failure case, suggest to check the kernel logs
When mounting fails, the node-plugin should give a suggestion to check the
kernel logs so that users can report problems better.
Edited the existing log to include the message in both rbd and cephfs.
Fixes: https://github.com/ceph/ceph-csi/issues/1006

Signed-off-by: Mudit Agarwal <muagarwa@redhat.com>
2020-05-27 10:52:37 +00:00
Yug Gupta
753f6b7fec cleanup: fix golint warnings in util, e2e
util: golint warns about exported methods to have a
comment or to unexport them.

e2e: golint warns about package comment to be of the form
"Package e2e ..."

Reported-by: https://goreportcard.com/report/github.com/ceph/ceph-csi

Updates: 

Signed-off-by: Yug Gupta <ygupta@redhat.com>
2020-05-20 13:18:41 +00:00
Niels de Vos
6cea9e2649 util: remove unused getPools()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-05-20 08:08:33 +00:00
Niels de Vos
3fea4fa827 util: remove unused context.Context from GetPoolName()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-05-20 08:08:33 +00:00
Niels de Vos
ec61f0746b util: implement GetPoolName() with go-ceph
Remove an other call to the "rados" executable and re-use the existing
connection to the Ceph cluster.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-05-20 08:08:33 +00:00
Niels de Vos
be8c3c4b72 cleanup: return InvalidPoolID on error in GetPoolID()
InvalidPoolID has recently been added, and can be used in other location
too. As GetPoolID is updated with this patch set, return InvalidPoolID
on errors too.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-05-20 08:08:33 +00:00
Niels de Vos
08144df2a8 util: remove unused context.Context parameter from GetPoolID()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-05-20 08:08:33 +00:00
Niels de Vos
c9b4ee42e0 util: use go-ceph instead of "rados" command for GetPoolID()
GetPoolID() did not return ErrPoolNotFound in case the pool could not be
found. This has been addressed as well, so that looking for an existing
pool behaves the same for checking by Name or ID.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-05-20 08:08:33 +00:00
John Mulligan
8dea42a624 journal: rename CSIJournal to Config
This better reflects what it actually does now.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-05-15 15:14:48 +00:00
John Mulligan
52603d595a journal: split journal types creating a new Connection type
Before, the one CSIJournal type was handling both configuration and
providing methods to make changes to the journal. This created the
temptation to modify the state of the global configuration object to
enact changes through the method calls.

This change creates a new type `journal.Connection` that takes the
monitors and credentials to create a short(er)-lived object to actually
read and make changes on the journal. This also avoid mixing the
arguments needed to connect to the cluster with the arguments needed
for the various journal read & update calls.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-05-15 15:14:48 +00:00
Yug Gupta
2cdf5c3b9f util: make util.ClusterInfo usable outside util package
functions like getClusterInfo() returns struct
instead of a set of strings.

Fix: 

Signed-off-by: Yug Gupta <ygupta@redhat.com>
2020-05-14 19:43:21 +00:00
Niels de Vos
63c458bd63 rbd: use rbdVolume.open() for metadata operations
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-05-14 18:20:15 +00:00
Niels de Vos
7a18e68a6e rbd: add rbdVolume.open() to get access to an image
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-05-14 18:20:15 +00:00
Niels de Vos
f889dc20bd cleanup: use switch/case for genVolFromVolID() error checking
By using switch/case it is easier to follow the error checking of the
genVolFromVolID() function. In case a new error is added as a return of
the function, it will be simpler to add checking for it.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-05-14 14:43:05 +00:00
John Mulligan
f091374ba9 journal: remove SetNamespace setter function
The SetNamespace setter function was called only once, immediately after
the creation of a volume journal object in cephfs only.
Remove this function so that it is no longer implied that this field can
be mutated after the journal is created. In it's place, use an extended
"constructor" NewCSIVolumeJournalWithNamespace that takes a namespace
value at create-time only.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-05-12 17:57:36 +00:00
John Mulligan
78a6de2bd0 journal: remove SetCSIDirectorySuffix to simplify journal creation
The function SetCSIDirectorySuffix was used only one per (long-lived,
gloabl) journal object. It is simpler to construct the journal objects
with this needed parameter:
1. As it is required to function and non-optional AFAICT
2. Removes the temptation to mutate global object
3. Reduces LOC with exact same functionality
4. SetCSIDirectorySuffix would not behave correctly if called a 2nd time
   anyway.

Point 4. means that if you called the function twice to change the
suffix when you previously had "csi.volumes.alice", you'd get
"csi.volumes.alice.bob" instead of "csi.volumes.bob" what one would
expect.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-05-12 08:19:09 +00:00
Madhu Rajanna
b065726f19 cephfs: use new 'ceph fs resize' command when available
Use ceph fs resize command when its available
in the cluster.if its not available fallback to
old style of resizing the subvolume

ceph change log:https://docs.ceph.com/docs/master/releases/nautilus/#v14-2-8-nautilus

Fixes 

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-05-12 06:57:44 +00:00
Niels de Vos
43857a3b75 rbd: implement deleteImage() with go-ceph
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-05-06 14:55:47 +00:00
Madhu Rajanna
22a86c568e rbd: default nouuid if the formattype is xfs
The problem happens when multiple PVCs with the
same UUID are attached/mounted on a node. This
can happen after creating a PVC from a snapshot,
or cloning a PVC.

make nouuid as the default mount option if
the format type is xfs to avoid mounting
issues.

updates: 

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-05-05 07:07:16 +00:00
Niels de Vos
45d1404d3e rbd: split buildCreateVolumeResponse() from CreateVolume()
The gocyclo linter complains about the high complexity of the
CreateVolume() function:

> pkg/rbd/controllerserver.go:133:1: cyclomatic complexity 21 of func `(*ControllerServer).CreateVolume` is high (> 20) (gocyclo)

By splitting it up and separeting the creation of an exisint CSI Volume
object in buildCreateVolumeResponse(), the gocyclic linter does not
complain any longer.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-05-04 13:21:03 +00:00
Niels de Vos
c89c68e9ea rbd: use 'rv' as name of the rbdVolume (golint)
golint has a pretty struct stylechek, it down not allow different
variable names for methods on an object:

    pkg/rbd/rbd_util.go:970:1: receiver name rbdVol should be consistent with previous receiver name rv for rbdVolume (golint)
    func (rbdVol *rbdVolume) ensureEncryptionMetadataSet(ctx context.Context) error {
    ^
    pkg/rbd/rbd_journal.go:166:26: ST1016: methods on the same type should have the same receiver name (seen 2x "rbdVol", 3x "rv") (stylecheck)
    func (rbdVol *rbdVolume) Exists(ctx context.Context) (bool, error) {
                             ^

Rename the 'rbdVol' variable to 'rv' to make it consistent.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-05-04 13:21:03 +00:00
Niels de Vos
f814bd72e5 rbd/go-ceph: add GetMetadata() and GetMetadata() functions
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-05-04 13:21:03 +00:00
Niels de Vos
ea51b04017 rbd: setup connection in genVolFromVolID()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-05-04 13:21:03 +00:00
Niels de Vos
12130123ac rbd: do not pass Credentials to checkVolExists()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-05-04 13:21:03 +00:00
Niels de Vos
805f10fd71 rbd: remove unused rbdVolume.open()
rbdVolume.open() was split from commit 5dd34732e1e while moving part of
the functionality to util.ClusterConnection. It seems that .open() is
not used anywhere at the moment, so drop it until follow-up patches
require it again.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-05-04 13:21:03 +00:00
Niels de Vos
01edaf8a71 move rbdVolume connection details to utils.ClusterConnection
The shared util.ClusterConnection can be used for rbd.rbdVolume and
cephfs.volumeOptions to connect to the Ceph cluster. This will then use
the shared ConnPool, and functions for obtaining connection details will
be the same across cephfs and rbd packages.

The ClusterConnection.Creds credentials are temporarily available until
all the functions have been adapted to use go-ceph and the connection
from the ConnPool.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-05-04 13:21:03 +00:00
Mathias Merscher
0991cdf498 make CephFS SubvolumeGroup configurable
The name of the CephFS SubvolumeGroup for the CSI volumes was hardcoded to "csi". To make permission management in multi tenancy environments easier, this commit makes it possible to configure the CSI SubvolumeGroup.

related to  and 
2020-05-04 05:50:06 +00:00
Yug Gupta
47226ccdf7 util: fix golint warnings in csiconfig, volid
golint warns about the following statements:

ceph-csi/internal/util/csiconfig.go
Line 49: warning: exported function Mons should have comment or be unexported (golint)

ceph-csi/pkg/util/volid.go :
Line 72: warning: exported method CSIIdentifier.ComposeCSIID should have comment
or be unexported (golint)

Reported-by: https://goreportcard.com/report/github.com/ceph/ceph-csi
Updates: 

Signed-off-by: Yug Gupta <ygupta@redhat.com>
2020-04-29 11:36:53 +00:00
John Mulligan
c8271fe64c journal: move voljournal.go to a new package
This new journal package isolates journal logic from the rest of util
and helps draw bright lines between what is a generic utility function
and what is csi journal logic.

Done partly as preparation for making use of go-ceph in journal.

No functional changes are made except to update references to allow the
code to compile.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-24 07:36:38 +00:00
John Mulligan
22d1476bba util: create a NewErrSnapNameConflict function
The NewErrSnapNameConflict will allow packages outside of "util" to
create new instances of the ErrSnapNameConflict error.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-24 07:36:38 +00:00
Niels de Vos
32839948ef cleanup: move pkg/ to internal/
The internal/ directory in Go has a special meaning, and indicates that
those packages are not meant for external consumption. Ceph-CSI does
provide public APIs for other projects to consume. There is no plan to
keep the API of the internally used packages stable.

Closes: 
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-04-23 11:00:59 +00:00