Commit Graph

35 Commits

Author SHA1 Message Date
Niels de Vos
16cb43f0f9 rbd: store csi.storage.k8s.io/pvc/namespace metadata as Owner
The Owner of an RBD image (Kubernetes Namespace, tenant) can be used to
identify additional configuration options. This will be used for
fetching the right Vault Token when encrypting/decrypting volumes.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-09 13:58:48 +00:00
Madhu Rajanna
8ebb9a1ba0 cleanup: fix misspell words
fixed misspell words detected by  codespell

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-11-30 08:46:48 +01:00
Madhu Rajanna
39b1f2b4d3 cleanup: fix mispell words
fixed mispell words in the repo.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-11-29 12:47:46 +05:30
Madhu Rajanna
68bd44beba rbd: add new controller to regenerate omap data
In the case of Disaster Recovery failover, the
user expected to create the static PVC's. We have
planned not to go with the PVC name and namespace
for many reasons (as in kubernetes it's planned to
support PVC transfer to a new namespace with a
different name and with new features coming in
like data populator etc). For now, we are
planning to go with static PVC's to support
async mirroring.

During Async mirroring only the RBD images are
mirrored to the secondary site, and when the
user creates the static PVC's on the failover
we need to regenerate the omap data. The
volumeHandler in PV spec is an encoded string
which contains clusterID and poolID and image UUID,
The clusterID and poolID won't remain same on both
the clusters, for that cephcsi need to generate the
new volume handler and its to create a mapping
between new volume handler and old volume handler
with that whenever cephcsi gets csi requests it
check if the mapping exists it will pull the new
volume handler and continues other operations.

The new controller watches for the PVs created,
It checks if the omap exists if it doesn't it
will regenerate the entire omap data.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-11-28 18:50:00 +00:00
Madhu Rajanna
ba84f14241 journal: create object with provided UUID
incase of async mirroring the volume UUID is
retrieved from the volume name, instead of cephcsi
generating a new UUID it should reserve the passed
UUID it will be useful when we support both metro DR
and async mirroring on a kubernetes clusters.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-11-28 18:50:00 +00:00
John Mulligan
8a41cd03a5 journal: fix reading omaps from objects with large key counts
The implementation of getOMapValues assumed that the number of key-value
pairs assigned to the object would be close to the number of keys
being requested. When the number of keys on the object exceeded the
"listExcess" value the function would fail to read additional keys
even if they existed in the omap.
This change sets a large fixed "chunk size" value and keeps reading
key-value pairs as long as the callback gets called and increments
the numKeys counter.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-11-06 06:42:22 +00:00
Humble Chirammal
75c6cb6544 rbd: remove unused credentials argument from StoreImageID()
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-10-15 06:46:41 +00:00
Madhu Rajanna
ed289fea5e journal: replace klog with util logger in voljournal.go
journal: replace klog with util logger in voljournal.go

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-19 14:38:12 +00:00
Madhu Rajanna
1a9cd23f64 journal: replace klog with util logger in omap.go
replace klog with util logger in omap.go

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-19 14:38:12 +00:00
Mehdy Khoshnoody
fc5eadf106 rbd: Add rados namespace support for rbd
Make sure to operate within the namespace if any given
when dealing with rbd images and snapshots and their journals.

Signed-off-by: Mehdy Khoshnoody <mehdy.khoshnoody@gmail.com>
2020-08-12 16:22:58 +05:30
Humble Chirammal
a4f2c5617c cephfs: retrieve NewCSISnapshotJournal with namespace set
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-08 00:11:35 +00:00
Sven Anderson
8393fbe40b util: simplify error handling
The sentinel error code had additional fields in the errors, that are
used nowhere.  This leads to unneccesarily complicated code.  This
change replaces the sentinel errors in utils with standard errors
created with errors.New() and adds a simple JoinErrors() function to
be able to combine sentinel errors from different code tiers.

Related: #1203

Signed-off-by: Sven Anderson <sven@redhat.com>
2020-07-23 11:16:40 +00:00
Yug
71ddf51544 cleanup: address gomnd warnings
Direct usage of numbers should be avoided.

Issue reported:
mnd: Magic number: X, in <argument> detected (gomnd)

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-07-21 08:36:24 +00:00
Yug
7f94a57908 cleanup: address godot warnings
Top level comments should end in a period

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-07-21 08:36:24 +00:00
Yug
1490daed7e cleanup: Avoid usage of numbers
Add seperate functions to handle all
levels and types of logging.

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-07-10 07:41:23 +00:00
Yug
8dc4ab6b1b rebase: update k8s.io/klog to v2.3.0
Update klog version to v2.3.0

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-07-10 07:41:23 +00:00
Madhu Rajanna
f05c8ed2a7 journal: remove unused rados CLI functions
Removed unused rados CLI functions, as rados
CLI is replaced with go-ceph bindings.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-06 15:03:44 +00:00
Madhu Rajanna
9bb79fbe7d journal: replace getStoredImageId with go-ceph
moved implementation of getStoredImageID to go-ceph
to get better performance.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-06 15:03:44 +00:00
Madhu Rajanna
341a3c9f59 journal: replace rados CLI with go-ceph to storeimage id
Moved the implementation of storeImageID from rados
CLI to go-ceph api.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-06 15:03:44 +00:00
Sven Anderson
de74c36d8c cleanup: address err113 warnings about direct error comparisons
Several places in the code compared errors directly with the go-ceph
sentinel errors.  This change uses the errors.Is() function of go
1.13 instead.  The err113 linter reported this issue as:

err113: do not compare errors directly, use errors.Is() instead

Signed-off-by: Sven Anderson <sven@redhat.com>
2020-07-06 08:57:46 +00:00
Niels de Vos
91ade42a32 cleanup: use errors.As() in journal.Connection methods
See-also: https://github.com/golang/go/wiki/ErrorValueFAQ#how-should-i-change-my-error-handling-code-to-work-with-the-new-features
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-03 09:12:48 +00:00
Niels de Vos
92aae4834e cleanup: use standard Golang errors package
"github.com/pkg/errors" does not offer more functionlity than that we
need from the standard "errors" package. With Golang v1.13 errors can be
wrapped with `fmt.Errorf("... %w", err)`. `errors.Is()` and
`errors.As()` are available as well.

See-also: https://tip.golang.org/doc/go1.13#error_wrapping
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-03 09:12:48 +00:00
Madhu Rajanna
6ba0cd58b9 rbd: add code to store/retrieve the image ID
cephcsi need to store and retrieve the rbd image ID
in the omap as we need the image ID to add a
task to remove the image from the Trash.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-07-01 08:21:47 +00:00
John Mulligan
09e1c856d0 journal: do not return errors from remove omap func if omap missing
The previous function used to remove omap keys apparently did not
return errors when removing omap keys from a missing omap (oid).
Mimic that behavior when using the api.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-06-22 15:41:00 +00:00
John Mulligan
c14cbee132 journal: change omap set func to handle multiple key-value pairs
For any function that sets more than one key on a single oid setting
them as a batch will be more efficient.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-06-22 15:41:00 +00:00
John Mulligan
a8409eccc8 journal: change omap remove func to handle multiple keys at once
For any function that removes more than one key on a single oid removing
them as a batch will be more efficient.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-06-22 15:41:00 +00:00
John Mulligan
0ac5f40d09 journal: change omap get func to handle multiple keys at once
Taking this appraoch means that any function that must get more than one
key's value from the same oid can be more efficient by calling out to
ceph only once.

To be cautious and avoid missing things we always request ceph return
more keys than we actually expect to be set on the oid. If there are
unexpected keys there, we will not miss the keys we want if we first hit
an unexpected key if we were to limit ourselves to iterating only over
the number of keys we're expecting to be on the object.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-06-22 15:41:00 +00:00
John Mulligan
cd24bb3f5c journal: convert journal to use new omap functions
Convert the business-logic of the journal to use the new go-ceph based
omap manipulation functions.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-06-22 15:41:00 +00:00
John Mulligan
0b99bdaa82 journal: add omap.go with go-ceph based omap manipulation funcs
These new omap manipulation functions (get/set/remove) are roughly
equivalent to the previous command-line based approach but rely
on direct api calls to ceph.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-06-22 15:41:00 +00:00
Niels de Vos
3fea4fa827 util: remove unused context.Context from GetPoolName()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-05-20 08:08:33 +00:00
John Mulligan
8dea42a624 journal: rename CSIJournal to Config
This better reflects what it actually does now.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-05-15 15:14:48 +00:00
John Mulligan
52603d595a journal: split journal types creating a new Connection type
Before, the one CSIJournal type was handling both configuration and
providing methods to make changes to the journal. This created the
temptation to modify the state of the global configuration object to
enact changes through the method calls.

This change creates a new type `journal.Connection` that takes the
monitors and credentials to create a short(er)-lived object to actually
read and make changes on the journal. This also avoid mixing the
arguments needed to connect to the cluster with the arguments needed
for the various journal read & update calls.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-05-15 15:14:48 +00:00
John Mulligan
f091374ba9 journal: remove SetNamespace setter function
The SetNamespace setter function was called only once, immediately after
the creation of a volume journal object in cephfs only.
Remove this function so that it is no longer implied that this field can
be mutated after the journal is created. In it's place, use an extended
"constructor" NewCSIVolumeJournalWithNamespace that takes a namespace
value at create-time only.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-05-12 17:57:36 +00:00
John Mulligan
78a6de2bd0 journal: remove SetCSIDirectorySuffix to simplify journal creation
The function SetCSIDirectorySuffix was used only one per (long-lived,
gloabl) journal object. It is simpler to construct the journal objects
with this needed parameter:
1. As it is required to function and non-optional AFAICT
2. Removes the temptation to mutate global object
3. Reduces LOC with exact same functionality
4. SetCSIDirectorySuffix would not behave correctly if called a 2nd time
   anyway.

Point 4. means that if you called the function twice to change the
suffix when you previously had "csi.volumes.alice", you'd get
"csi.volumes.alice.bob" instead of "csi.volumes.bob" what one would
expect.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-05-12 08:19:09 +00:00
John Mulligan
c8271fe64c journal: move voljournal.go to a new package
This new journal package isolates journal logic from the rest of util
and helps draw bright lines between what is a generic utility function
and what is csi journal logic.

Done partly as preparation for making use of go-ceph in journal.

No functional changes are made except to update references to allow the
code to compile.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-04-24 07:36:38 +00:00