Commit Graph

175 Commits

Author SHA1 Message Date
Madhu Rajanna
bfde065f92 util: move getMonsAndClusterID to util
as we had duplicate functions in both cephfs
and rbd this commit moves the function to util.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-11 10:02:53 +00:00
Madhu Rajanna
18f4e9d519 util: move csiconfigfile to util
as we have csiconfigfile in both cephfs
and rbd moving the configfile path to
util folder.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-11 10:02:53 +00:00
Niels de Vos
b864afd6d7 util: use local ErrorLog() for Create/RemoveObject() instead of klog
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-08-11 08:11:37 +00:00
Niels de Vos
eaea18df0f util: use local FatalLog() for NewK8sClient() instead of klog
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-08-11 08:11:37 +00:00
Niels de Vos
9d4020d255 util: use local ErrorLog() for CheckKernelSupport() instead of klog
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-08-11 08:11:37 +00:00
Niels de Vos
75ae16cfab util: use local DefaultLog() for util/topology instead of klog
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-08-11 08:11:37 +00:00
Niels de Vos
bf02227207 util: use local ErrorLog() for OperationLock.release() instead of klog
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-08-11 08:11:37 +00:00
Niels de Vos
fb56a7595a util: use local ErrorLog() for GetCryptoPassphrase() instead of klog
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-08-11 08:11:37 +00:00
Niels de Vos
61924adf57 util: use FatalLog() in util/httpserver.go
Hide the use of klog by using our own FatalLog() function.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-08-11 08:11:37 +00:00
Niels de Vos
ab033f85df util: add FatalLog(), ErrorLog() and WarningLog() functions
These functions will always log the message, irrespectively of the
log-level that has been configured.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-08-11 08:11:37 +00:00
Niels de Vos
44863a9d29 cleanup: move log functions to util/log.go
There is a util/log.go file, so the log functions in util/util.go can be
moved there.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-08-11 08:11:37 +00:00
Madhu Rajanna
b86d329d1e rbd: add stagingpath to error message
if the stagingpath is not present inside
the container send the staging path in error
message.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-10 08:51:23 +00:00
Humble Chirammal
7d2bba1375 cephfs: introduce parsetime() to parse createdAt field in snap return
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-08 00:11:35 +00:00
Humble Chirammal
c3400bfb97 cephfs: add snapshot create and delete functionalilies
It also add helper routines like parsetime,doSnapshot..etc

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-08 00:11:35 +00:00
Humble Chirammal
20c90ddfc4 cephfs: getVolumeRootPathCeph and purgeVolume use new error strings
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-08 00:11:35 +00:00
Humble Chirammal
1f4839cb5e cephfs: add subvolume struct and getSubvolumeInfo calls
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-08 00:11:35 +00:00
Humble Chirammal
31e9288f8f cephfs: remove inValidCommmand and errNotFoundString
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-08 00:11:35 +00:00
Humble Chirammal
5bceb590fd cephfs: add snap reserve/unreserve and snap exist functionalities
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-08 00:11:35 +00:00
Humble Chirammal
38d005e4e6 cephfs: validate create volume request
Also add functionality to generate snap from request
and to get mon and clusterID from the request

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-08 00:11:35 +00:00
Humble Chirammal
9c000add29 cephfs: Change checkVolExist for snapshot and clone workflow
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-08 00:11:35 +00:00
Humble Chirammal
c773097f85 cephfs: add snapshot and clone helper functions
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-08 00:11:35 +00:00
Humble Chirammal
d1fe12b4f0 cephfs: change createvolume for snapshot/clone workflow
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-08 00:11:35 +00:00
Humble Chirammal
c31f349c6f cephfs: use delete operation lock in delete volume
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-08 00:11:35 +00:00
Humble Chirammal
f9257ed1cb cephfs: add snapshot create/delete capabilities in controllerserver
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-08 00:11:35 +00:00
Humble Chirammal
5a71949dcd cephfs: introduce newSnapshotOptionsFromID to generate volOpt and sid
From provided CSI volume ID this populate volumeOptions and snapshot
identifier after connecting to the snapJournal.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-08 00:11:35 +00:00
Humble Chirammal
a4f2c5617c cephfs: retrieve NewCSISnapshotJournal with namespace set
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-08 00:11:35 +00:00
Humble Chirammal
1746b4e6ee cephfs: add various error strings for command validation
Compared to previous version of the error strings, this change
depend on error strings like ENOENT, EEXIST, EINVAL..etc
The format of the error strings change in different cluster versions
and the error code return should not change. This also add extra
error strings for snapshot and clone operation outputs

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-08 00:11:35 +00:00
Humble Chirammal
6dcf03a4bc cephfs: add checkContentSource() to validate the data source
checkContentSource() validate the data source in the request
and then populate volumeOptions or snapshotshot identifier in
case of snapshot source. If the data source is volume, then
parentVolumeOption and pvID are populated.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-08 00:11:35 +00:00
Humble Chirammal
2bb0bbde7f cephfs: change createBackingVolume to accomodate clone operations
adjust createBackingVolume() to create a subvolume from snapshot
or existing subvolume by taking restore or clone operation locks

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-08 00:11:35 +00:00
Humble Chirammal
c968380134 cephfs: add snapshotlock and operationlock to controllerserver
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-08 00:11:35 +00:00
Humble Chirammal
66827facc3 cephfs: make use of expand operation lock in ControllerExpandVolume
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-08 00:11:35 +00:00
Madhu Rajanna
2458ec6573 rbd: return error if fetching cluster id fails
if we are not able to fetch the cluster-ID from
the createSnapshot request and also if we are
not able to get the monitor information from
the cluster-ID return error instead of using
the parent image information.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-03 14:25:06 +00:00
Mudit Agarwal
9ed0811422 rbd: implement rbdVolume.resize() with go-ceph
Replaced command execution with go-ceph Resize() function.
Volsize is being updated before waiting for resize() to return,
fixed it to get updated only after resize() is successful.

Signed-off-by: Mudit Agarwal <muagarwa@redhat.com>
2020-08-03 10:50:01 +00:00
Niels de Vos
be9e7cf956 rbd: pass context.Context to rbdVolume.resize()
While adding the context.Context to the resizeRBDimage() function, it
became a little ugly. So renaming the function to resize() and making it
a method of the rbdVolume type.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-24 16:04:13 +00:00
Niels de Vos
36469b87e2 util: make ExecComand return stdout and stderr as string
Most consumers of util.ExecCommand() need to convert the returned []byte
format of stdout and/or stderr to string. By having util.ExecCommand()
return strings instead, the code gets a little simpler.

A few commands return JSON that needs to be parsed. These commands will
be replaced by go-ceph implementations later on. For now, convert the
strings back to []byte when needed.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-24 16:04:13 +00:00
Niels de Vos
ddac66d76b util: use context.Context for logging in ExecCommand
All calls to util.ExecCommand() now pass the context.Context. In some
cases this is not possible or needed, and util.ExecCommand() will not
log the command.

This should make debugging easier when command executions fail.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-24 16:04:13 +00:00
Niels de Vos
bb4f1c7c9d rbd: use util.ExecCommand() instead of execCommand()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-24 16:04:13 +00:00
Niels de Vos
457d846241 cephfs: use util.ExecCommand() instead of execCommand()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-24 16:04:13 +00:00
Niels de Vos
47d5b60af8 rbd: disable reflink while creating XFS filesystems
Current versions of the mkfs.xfs binary enable reflink support by
default. This causes problems on systems where the kernel does not
support this feature. When the kernel the feature does not support, but
the filesystem has it enabled, the following error is logged in `dmesg`:

    XFS: Superblock has unknown read-only compatible features (0x4) enabled

Introduce a check to see if mkfs.xfs supports the `-m reflink=` option.
In case it does, pass `-m reflink=0` while creating the filesystem.

The check is executed once during the first XFS filesystem creation. The
result of the check is cached until the nodeserver restarts.

Fixes: #966
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-24 13:37:51 +00:00
Niels de Vos
526da43b6a rbd: remove unused rbdStatus()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-24 11:34:48 +00:00
Niels de Vos
7afaac9c66 rbd: implement rbdVolume.isInUse() with go-ceph
The new rbdVolume.isInUse() method will replace the rbdStatus()
function. This removes one more rbd command execution in the
DeleteVolume path.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-24 11:34:48 +00:00
Sven Anderson
92884f56f4 rbd: simplify error handling
This change replaces the sentinel errors in rbd module with
standard errors created with errors.New().

Related: #1203

Signed-off-by: Sven Anderson <sven@redhat.com>
2020-07-23 11:16:40 +00:00
Sven Anderson
dba2c27bcb cephfs: simplify error handling
This change replaces the sentinel errors in cephfs module with
standard errors created with errors.New().

Related: #1203

Signed-off-by: Sven Anderson <sven@redhat.com>
2020-07-23 11:16:40 +00:00
Sven Anderson
7c9c7c78a7 util: add tests for JoinErrors()
Signed-off-by: Sven Anderson <sven@redhat.com>
2020-07-23 11:16:40 +00:00
Sven Anderson
8393fbe40b util: simplify error handling
The sentinel error code had additional fields in the errors, that are
used nowhere.  This leads to unneccesarily complicated code.  This
change replaces the sentinel errors in utils with standard errors
created with errors.New() and adds a simple JoinErrors() function to
be able to combine sentinel errors from different code tiers.

Related: #1203

Signed-off-by: Sven Anderson <sven@redhat.com>
2020-07-23 11:16:40 +00:00
Madhu Rajanna
b3a4f510e6 rbd: take operation locks before operating on resource
Take operation locks on the resources before operating
on the resouces. This allows us to do parallel operations
for some RPC calls such as Clone and Restore of PVC.
This operations will only be blocked if the image is
expanding or Snapshot and RBD image is getting deleted.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-21 13:25:19 +00:00
Madhu Rajanna
d6348545ab journal: Add additional operation based locking
As we are adding new functionalities like Create/Delete
snapshot,Clone from Snapshot and Clone from Volume.
with the current implementation, there are only serial
operations allowed for this functionalities, for some
function we can allow parallel operations like
Clone from snapshot and Clone from Volume and Create
`N` snapshots on a single volume.

Delete Volume: Need to ensure that there is no clone,
Snapshot create and  Expand volume in progress.

Expand Volume: Need to ensure that there is no clone,
snapshot create and cloning in progress

Delete Snapshot: Need to ensure that there is no
cloning in progress

Restore Volume/Snapshot: Need to ensure that there is
no Expand or delete operation in progress.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-21 13:25:19 +00:00
Yug
71ddf51544 cleanup: address gomnd warnings
Direct usage of numbers should be avoided.

Issue reported:
mnd: Magic number: X, in <argument> detected (gomnd)

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-07-21 08:36:24 +00:00
Yug
e73fe64a0d cleanup: address gosec warnings
gosec warns about security problems by scanning the
Go AST.

Issues Reported:
G101 (CWE-798): Potential hardcoded credentials (Confidence: LOW, Severity: HIGH)
G204 (CWE-78): Subprocess launched with variable (Confidence: HIGH, Severity: MEDIUM)
G304 (CWE-22): Potential file inclusion via variable (Confidence: HIGH, Severity: MEDIUM)

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-07-21 08:36:24 +00:00
Yug
48fa43270f cleanup: address gocritic warnings
Add explanation to nolint directives.

Issue reported:
whyNoLint: include an explanation for nolint directive (gocritic)

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-07-21 08:36:24 +00:00