Commit Graph

306 Commits

Author SHA1 Message Date
Niels de Vos
d2c4cacb39 rbd: restart thick-provisioned PVC snapshot restoring after aborting
In case restoring a snapshot of a thick-PVC failed during DeepCopy(),
the image will exist, but have partial contents. Only when the image has
the thick-provisioned metadata set, it has completed DeepCopy().

When the metadata is missing, the image is deleted, and an error is
returned to the caller. Kubernetes will automatically retry provisioning
on the ABORTED error, and the restoring will get restarted from the
beginning.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-06-23 14:22:28 +00:00
Niels de Vos
7f1bdb49d1 rbd: use DeepCopy() when restoring a thick-snapshot
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-06-23 14:22:28 +00:00
Madhu Rajanna
591ba3f580 rbd: set thick provision metadata on clone volume
the parent volume(CreateVolume) and the clone volume
(CreateSnapshot) are both indepedent and parent volume
can be deleted anytime. To check the thick provision
during Snapshot restore(CreateVolume from snapshot)
we need the thick provision metadata so for the same
reason setting the thick provision metadata on the
clone image we are creating at the CreateSnapshot time.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-06-18 10:57:48 +00:00
Madhu Rajanna
6d14eeee70 rbd: use RbdSnapName to check the image details
RbdSnapName holds the actual RBD image name which
got created during the CreateSnapshot operation.
RbdImageName holds the name of the parent from
which the snapshot is created. and the parent
is independent of snapshot and it can be deleted
any time for the same reason using the RbdSnapName
to check the rbd image details.

generate a temporary volume from the snapshot which
replaces the rbdImageName with RbdSnapName and use
it to check the image metadata.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-06-18 10:57:48 +00:00
Madhu Rajanna
7966d2e5c1 rbd: add validation for thick restore/clone
added validation to allow only Restore of Thick PVC
snapshot to a thick clone and creation of thick clone
from thick PVC.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-06-18 10:57:48 +00:00
Madhu Rajanna
fc442221e4 rbd: make isThickProvisioned method of rbdImage
isThickProvisioned can be used for both snapshot
and clone validation if isThickProvisioned is method
of common rbdImage structure.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-06-18 10:57:48 +00:00
Niels de Vos
57d3183cb1 rbd: restart thick-provisioned PVC cloning after aborting
In case cloning a thick-PVC failed during DeepCopy(), the image will
exist, but have partial contents. Only when the image has the
thick-provisioned metadata set, it has completed DeepCopy().

When the metadata is missing, the image is deleted, and an error is
returned to the caller. Kubernetes will automatically retry provisioning
on the ABORTED error, and the cloning will get restarted from the
beginning.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-06-18 06:25:56 +00:00
Niels de Vos
b1045364d9 rbd: disable FeatureDeepFlatten when doing DeepCopy()
Not all Linux kernels support the deep-flatten feature. Disabling the
feature makes it possible to map RBD images on older kernels (like what
minikube uses).

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-06-18 06:25:56 +00:00
Niels de Vos
4908ff8743 rbd: no need to flatten thick-provisioned images
Thick-provisioned images are independent, cloned images or snapshots are
deep-flattened during creation. There is no need to try and flatten them
again.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-06-18 06:25:56 +00:00
Niels de Vos
6cc11c15d3 rbd: use DeepCopy to create a thick-provisioned clone
To create a full-allocated RBD image from a snapshot/clone DeepCopy()
can be used. This is needed when the parent of the new volume is
thick-provisioner, so that the new volume is independent of the parent
and thick-provisioned as well.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-06-18 06:25:56 +00:00
Niels de Vos
334f237e23 cleanup: move snapshot/clone/flatten into its own function
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-06-18 06:25:56 +00:00
Madhu Rajanna
367eb9f748 rbd: correct return error for isCompatibleEncryption
isCompatibleEncryption is used to validate the
requested volume and the existing volume and
the destination volume name wont be generated yet
and logging the destination volume prints the empty
image name with pool name.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-06-17 10:12:18 +00:00
Madhu Rajanna
05b8433b89 rbd: check stdErr for does not have a parent error
actual error will be present in the stdErr not the error
when we try to add a task to flatten the rbd image. This
commits corrects the error checking when the image does
not have a parent.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-06-15 11:07:34 +00:00
Yati Padia
6bfdf2feb0 cleanup: gocyclo being unused for linter
This commit addresses the following issue:
'nolint:gocyclo // complexity needs to be reduced.'
is unused for linter "gocyclo" (nolintlint)

Updates:#2025

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-06-15 02:54:16 +00:00
Yug
5c079894c7 doc: correct comment indentation in rbdVolume
correct comment indentation in rbdvolume{}

Signed-off-by: Yug <yuggupta27@gmail.com>
2021-06-15 02:34:51 +00:00
Yug
d992803e9e rbd: Update pool name in image chain
While traversing image chain, the parent
image can be present in a different pool
that the one child is in. So, updating
pool name in the next itteration to
that of the Parent.

Co-authored-by: Madhu Rajanna <madhupr007@gmail.com>
Signed-off-by: Yug <yuggupta27@gmail.com>
2021-06-10 21:46:53 +00:00
Yug
1f6a9cabfd rbd: verify if pool name is not empty
Validate Snapshot request to check if the
passed pool name is not empty.

Co-authored-by: Madhu Rajanna <madhupr007@gmail.com>
Signed-off-by: Yug <yuggupta27@gmail.com>
2021-06-10 21:46:53 +00:00
Yug
3898ae34a7 rbd: open new ioctx connection
if the parent and child clones are in
different namespaces we need to open a new
ioctx for pools.

Co-authored-by: Madhu Rajanna <madhupr007@gmail.com>
Signed-off-by: Yug <yuggupta27@gmail.com>
2021-06-10 21:46:53 +00:00
Yug
b63b0bf18d rbd: retrieve parent pool name of child image
when clones are created in different pool,we
need to retrieve the parent pool to get the
information of the parent image.

Co-authored-by: Madhu Rajanna <madhupr007@gmail.com>
Signed-off-by: Yug <yuggupta27@gmail.com>
2021-06-10 21:46:53 +00:00
Yug
e699318acc rbd: pass parent volume to undoSnapshotCloning function
as we are supporting the creation of clone to a new
pool we need to pass the correct parent volume
to cleanup the snapshot on parent volume.

Co-authored-by: Madhu Rajanna <madhupr007@gmail.com>
Signed-off-by: Yug <yuggupta27@gmail.com>
2021-06-10 21:46:53 +00:00
Yug
961c1d12fd rbd: add support to create clone in different pool
added support to create image in different pool.
if the snapshot/rbd image exists in one pool we
can create a clone the clone of the rbd image to
a different pool.

Co-authored-by: Madhu Rajanna <madhupr007@gmail.com>
Signed-off-by: Yug <yuggupta27@gmail.com>
2021-06-10 21:46:53 +00:00
Mohammed Naser
671d6a7767 rbd: Backout if image features is empty
In golang world, if you split an empty string that does not contain
the seperator, you get an array with one empty string.  This results
in volumes failing to mount with "invalid feature " (note extra space
because it's trying to check if 'empty string' is a valid feature).

This patch checks if the string is empty, and if so, it just decides
to skip the entire validation and returning nothing.

Signed-off-by: Mohammed Naser <mnaser@vexxhost.com>
2021-06-10 15:43:09 +00:00
Mohammed Naser
f193ebfbb1 rbd: Add failing test when no features are provided
Signed-off-by: Mohammed Naser <mnaser@vexxhost.com>
2021-06-10 15:43:09 +00:00
Madhu Rajanna
7b5c78ec7c rbd: fail fast in create volume for missmatch encryption
CreateVolume will fail in below cases

* If the snapshot is encrypted and requested volume
is not encrypted
* If the snapshot is not encrypted and requested
volume is encrypted

* If the parent volume is encrypted and requested volume
is not encrypted
* If the parent volume is not encrypted and requested
volume is encrypted

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-06-07 15:05:21 +00:00
Yati Padia
0f44c6acb7 cleanup: address wasted assign issues
At places variable is reassigned without
being used.

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-06-03 09:51:14 +00:00
Niels de Vos
7cbad9305f rbd: repair thick-provisioned images on CreateVolume restart
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-06-01 14:42:12 +00:00
Niels de Vos
96a8ea3e88 cleanup: split repairExistingVolume() from CreateVolume()
Move the repairing of a volume/snapshot from CreateVolume to its own
function. This reduces the complexity of the code, and makes the
procedure easier to understand. Further enhancements to repairing an
exsiting volume can be done in the new function.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-06-01 14:42:12 +00:00
Madhu Rajanna
2e978e4211 rbd: fix typo in error message
fixed typo in error message.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-06-01 10:40:07 +00:00
Rakshith R
b891e5585d cleanup: address ifshort linter issues
This commit addresses ifshort linter issues which
checks if short syntax for if-statements is possible.

updates: #1586

Signed-off-by: Rakshith R <rar@redhat.com>
2021-05-26 07:04:32 +00:00
Rakshith R
6618e2012d cleanup: remove unnecessary calling of .String() when logging
This commit removes calling of .String() when logging
since `%s`,`%v` or `%q` will call an existing .String() function
automatically.

Fixes: #2051

Signed-off-by: Rakshith R <rar@redhat.com>
2021-05-25 18:02:11 +00:00
Yati Padia
774e8e4042 util: enable golang profiling
Add support for golang profiling.
Standard tools like go tool pprof and curl
work. example:
$ go tool pprof http://localhost:8080/debug/pprof/profile
$ go tool pprof http://localhost:8080/debug/pprof/heap
$ curl http://localhost:8080/debug/pprof/heap?debug=1

https://golang.org/pkg/net/http/pprof/ contains
more details about the pprof interface.

Fixes: #1699

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-05-25 10:41:22 +00:00
Niels de Vos
25d0a1cfc0 rbd: add support for block-devices in NodeGetVolumeStats()
The NodeGetVolumeStats procedure can now be used to fetch the capacity
of the RBD block-device. By default this is a thin-provisioned device,
which means that the capacity is not reserved in the Ceph cluster. This
makes it possible to over-provision the cluster.

In order to detect the amount of storage used by the RBD block-device
(when thin-provisioned), it is required to connect to the Ceph cluster.
Unfortunately, the NodeGetVolumeStats CSI procedure does not provide
enough parameters to connect to the Ceph cluster and fetch more details
about the RBD image.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-05-25 06:41:04 +00:00
Madhu Rajanna
0ce6ad1152 rbd: fix image details logging
log only the required details of
the image.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-05-07 07:57:37 +00:00
Madhu Rajanna
67d73cd6e9 rbd: flatten image if the depth is not zero
flatten the image if the deep-flatten feature
is present on the images in the chain or if the
images in chain is not zero, as we cannot check
the deep-flatten feature the images which are
in trash.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-05-07 07:57:37 +00:00
Madhu Rajanna
e15e2e5081 rbd: discard image not found error
For flatten we call checkImageChainHasFeature
which internally calls to getImageInfo returns
the parent name even if the parent is in the trash,
when we try to open the parent image to get its
information it fails as the image not found.
we should treat error as nil if the parent is not found.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-05-07 07:57:37 +00:00
Niels de Vos
f11a041f56 cleanup: address gosec complaint about creating a file
The new gosec 2.7.0 complains like:

    G304 (CWE-22): Potential file inclusion via variable (Confidence: HIGH, Severity: MEDIUM)

Updates: #2025
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-05-05 16:05:23 +00:00
Madhu Rajanna
07a916b84d rbd: mark image ready when image state is up+unknown
To recover from split brain (up+error) state the image need to be
demoted and requested for resync on site-a and then the image on site-b
should gets demoted.The volume should be marked to ready=true when the
image state on both the clusters are up+unknown because during the last
snapshot syncing the data gets copied first and then image state on the
site-a changes to up+unknown.

If the image state on both the sites are up+unknown consider that
complete data is synced as the last snapshot
gets exchanged between the clusters.

* create 10 GB of file and validate the data after resync

* Do Failover when the site-a goes down
* Force promote the image and write data in GiB
* Once the site-a comes back, Demote the image and issue resync
* Demote the image on site-b
* The status will get reflected on the other site when the last
  snapshot sync happens
* The image will go to up+unknown state. and complete data will
  be copied to site a
* Promote the image on site-a and use it

```bash
csi-vol-5633715e-a7eb-11eb-bebb-0242ac110006:
  global_id:   e7f9ec55-06ab-46cb-a1ae-784be75ed96d
  state:       up+unknown
  description: remote image demoted
  service:     a on minicluster1
  last_update: 2021-04-28 07:11:56
  peer_sites:
    name: e47e29f4-96e8-44ed-b6c6-edf15c5a91d6-rook-ceph
    state: up+unknown
    description: remote image demoted
    last_update: 2021-04-28 07:11:41
 ```

* Do Failover when the site-a goes down
* Force promote the image on site-b and write data in GiB
* Demote the image on site-b
* Once the site-a comes back, Demote the image on site-a
* The images on the both site will go to split brain state

```bash
csi-vol-37effcb5-a7f1-11eb-bebb-0242ac110006:
  global_id:   115c3df9-3d4f-4c04-93a7-531b82155ddf
  state:       up+error
  description: split-brain
  service:     a on minicluster2
  last_update: 2021-04-28 07:25:41
  peer_sites:
    name: abbda0f0-0117-4425-8cb2-deb4c853da47-rook-ceph
    state: up+error
    description: split-brain
    last_update: 2021-04-28 07:25:26
```
* Issue resync
* The images cannot be resynced because when we issue resync
  on site a the image on site-b was in demoted state
* To recover from this state (promote and then demote the
  image on site-b after sometime)

```bash
csi-vol-37effcb5-a7f1-11eb-bebb-0242ac110006:
  global_id:   115c3df9-3d4f-4c04-93a7-531b82155ddf
  state:       up+unknown
  description: remote image demoted
  service:     a on minicluster1
  last_update: 2021-04-28 07:32:56
  peer_sites:
    name: e47e29f4-96e8-44ed-b6c6-edf15c5a91d6-rook-ceph
    state: up+unknown
    description: remote image demoted
    last_update: 2021-04-28 07:32:41
```
* Once the data is copied we can see that  the image state
  is moved to up+unknown on both sites
* Promote the image on site-a and use it

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-05-05 13:38:29 +00:00
Madhu Rajanna
c3bae17fce rbd: delete encryption key from KMS
when a Snapshot is encrypted during a CreateSnapshot
operation, the encryption key gets created in the KMS
when we delete the Snapshot the key from the KMS
should also gets deleted.

When we create a volume from snapshot we are copying
required information but we missed to copy the
encryption information, This commit adds the missing
information to delete the encryption key.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-04-30 08:05:47 +00:00
Humble Chirammal
798437d0c4 rbd: return crypt error for the rpc return
At present we return the volume connect error if the clone
from snapshot fails when rbdvolume is encrypted, which is incorrect.
This patch correctly return the failed copy encryption error to the
caller

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-04-21 16:10:20 +00:00
Madhu Rajanna
52290333e6 rbd: modified logic to check image watchers
Before RBD map operation, we do check the
watchers on the RBD image. In the case of
RWO volume. cephcsi makes sure only one
client is using the RBD image. If the rbd
image is mirrored, by default mirroring
daemon will add a watcher on the image
and as we are using go-ceph a watcher will
be added as we have opened the image So
we will have two watchers on an image if
mirroring is enabled. This holds when the
rbd mirror daemon is running, In case if
the mirror daemon is not running there will
be only one watcher on the rbd image
(which is placed by go-ceph image open)
we should not block the map operation if
the mirroring daemon is not running as
its Async mirroring. This commit adds a
check to make sure no more than 2 watchers
if the image is mirrored or no more than 1
watcher if it is not mirrored image.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-04-19 16:30:55 +00:00
Yug
6a46f381c2 cleanup: update description to generic
Since rbdImage is a common struct for
rbdVolume and rbdSnapshot, it description
was matching to only snapshot.
This commit makes the comments generic for
both volumes and snapshots.

Signed-off-by: Yug <yuggupta27@gmail.com>
2021-04-19 07:32:35 +00:00
Madhu Rajanna
eea52847bc rbd: check volumeID in PV if image not found
If the pool or few keys are missing in the omap.
GetImageAttributes function returns nil error message and few
empty items in imageAttributes struct. if the image is not
found and  the entiries are missing use
the volumeId present on the PV annotation for further operations.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-04-15 17:13:06 +05:30
Madhu Rajanna
cfc88c9910 rbd: discard up+unknown state in ResyncVolume
incase if the image is promoted and demoted the
image state will be set to up+unknown if the image
on the remote cluster is still in demoted state.

when user changes the state from primary to secondary
and still the image is in demoted (secondary) state
in the remote cluster. the image state on both the cluster
will be on unknown state.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-04-15 17:13:06 +05:30
Niels de Vos
8b8480017b logging: report issues in rbdImage.DEKStore API with stacks
It helps to get a stack trace when debugging issues. Certain things are
considered bugs in the code (like missing attributes in a struct), and
might cause a panic in certain occasions.

In this case, a missing string will not panic, but the behaviour will
also not be correct (DEKs getting encrypted, but unable to decrypt).
Clearly logging this as a BUG is probably better than calling panic().

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-04-14 03:59:28 +00:00
Niels de Vos
b1d05a1840 rbd: repair encryption config in case it is missing
It is possible that when a provisioner restarts after a snapshot was
cloned, but before the newly restored image had its encryption metadata
set, the new image is not marked as encrypted. This will prevent
attaching/mounting the image, as the encryption key will not be fetched,
or is not available in the DEKStore.

By actively repairing the encryption configuration when needed, this
problem should be addressed.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-04-14 03:59:28 +00:00
Niels de Vos
1482105309 cleanup: use buildCreateVolumeResponse() to simplify CreateVolume()
buildCreateVolumeResponse() exists exactly for the need to create a
csi.CreateVolumeResponse based on an rbdVolume. Calling this helper
reduces the code duplication in CreateVolume().

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-04-14 03:59:28 +00:00
Niels de Vos
52433841b4 cleanup: move copyEncryptionConfig() from CreateVolume to Exists()
The rbdVolume that needs its encryption configured is constructed in the
Exists() method. It is suitable to move the copyEncryptionConfig() call
there as well, so that the object is completely constructed in a single
place.

Golang-ci:gocyclo complained about the increased complexity of the
Exists() function. Moving the repairing of the ImageID into its own
helper function makes the code a little easier to understand.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-04-14 03:59:28 +00:00
Niels de Vos
596410ae60 cleanup: address "nolint" comments for RBD CreateSnapshot
Introduce helper function cloneFromSnapshot() that takes care of the
procedures that are needed when an existing snapshot has been found.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-04-14 03:59:28 +00:00
Niels de Vos
b5d0524c39 cleanup: release resources for rbdImages objects after use
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-04-14 03:59:28 +00:00
Niels de Vos
dc990037a5 rbd: move setupEncryption() from buildCreateVolumeResponse to CreateVolume
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-04-14 03:59:28 +00:00
Niels de Vos
bea9d56117 rbd: copyEncryptionConfig in doSnapshotClone()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-04-14 03:59:28 +00:00
Niels de Vos
fd5f4dbafd rbd: configureEncryption() in genSnapFromSnapID()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-04-14 03:59:28 +00:00
Niels de Vos
6fd3f57f40 rbd: set kmsID in reserveSnap()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-04-14 03:59:28 +00:00
Niels de Vos
0a046c5b6d rbd: copy encryption configuration in CreateSnapshot
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-04-14 03:59:28 +00:00
Niels de Vos
6b1285d38b rbd: copy passphrase for encrypted clones
When a source volume is encrypted, the passphrase needs to be copied and
stored for the newly cloned volume.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-04-14 03:59:28 +00:00
Niels de Vos
7b332a0184 rbd: add rbdImage.copyEncryptionConfig() to copy encryption metadata
Cloning volumes requires copying the DEK from the source to the newly
cloned volume. Introduce copyEncryptionConfig() as a helper for that.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-04-14 03:59:28 +00:00
Niels de Vos
b6aa19eea5 rbd: pass secrets when creating an source rbdVolume for cloning
Without this, the rbdVolume can not connect to the Ceph cluster and
configure the (optional) encryption.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-04-14 03:59:28 +00:00
Niels de Vos
92b2e08adf rbd: improve logging in deleteImage()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-04-14 03:59:28 +00:00
Niels de Vos
99da92cfd7 rbd: move deletion of DEK to deleteImage()
The ControllerServer should not need to care about support for
encryption, ideally it is transparantly handled by the rbdVolume type
and its internal API.

Deleting the DEK was one of the last remainders that was explicitly done
inside the ControllerServer.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-04-14 03:59:28 +00:00
Niels de Vos
a7c261a394 logging: correct formatting when reporting error in createVolumeFromSnapshot()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-04-14 03:59:28 +00:00
Madhu Rajanna
0f8813d89f rbd:store/Read volumeID in/from PV annotation
In the case of the Async DR, the volumeID will
not be the same if the clusterID or the PoolID
is different, With Earlier implementation, it
is expected that the new volumeID mapping is
stored in the rados omap pool. In the case of the
ControllerExpand or the DeleteVolume Request,
the only volumeID will be sent it's not possible
to find the corresponding poolID in the new cluster.

With This Change, it works as below

The csi-rbdplugin-controller will watch for the PV
objects, when there are any PV objects created it
will check the omap already exists, If the omap doesn't
exist it will generate the new volumeID and it checks for
the volumeID mapping entry in the PV annotation, if the
mapping does not exist, it will add the new entry
to the PV annotation.

The cephcsi will check for the PV annotations if the
omap does not exist if the mapping exists in the PV
annotation, it will use the new volumeID for further
operations.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-04-07 11:46:27 +00:00
Rakshith R
020cded581 cleanup: refactor deeply nested if statements in internal/rbd
Refactored deeply nested if statement in internal/rbd to
reduce cognitive complexity.

Signed-off-by: Rakshith R <rar@redhat.com>
2021-04-07 02:31:41 +00:00
Niels de Vos
aaeb35eceb rbd: encrypted volumes can be of type "crypto_LUKS" too
It seems that newer versions of some tools/libraries identify encrypted
filesystems with `crypto_LUKS` instead of `crypt`.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-04-06 15:54:27 +00:00
Madhu Rajanna
d7838defcf rbd: return FailedPrecondition error message
In case of the DR the image on the primary site cannot be
demoted as the cluster is down, during failover the image need
to be force promoted. RBD returns `Device or resource busy`
error message if the image cannot be promoted for above reason.
Return FailedPrecondition so that replication operator can send
request to force promote the image.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-04-06 14:12:41 +00:00
Madhu Rajanna
403532c9a6 rbd: use force from PromoteVolume Request
instead of fetching the force option from the
parameters. Use the Force field available in
the PromoteVolume Request.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-04-06 14:12:41 +00:00
Madhu Rajanna
385a751b8e rebase: rename kube-storage to csi-addons
as the org github.com/kube-storage is renamed
to github.com/csi-addons as the name kube-storage
was more generic.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-04-06 10:59:58 +00:00
Humble Chirammal
314fe0e23d cleanup: correct misspelling in rbd/clone.go
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2021-04-05 09:34:09 +00:00
Madhu Rajanna
448be70682 rbd: early check for disabled,disabling in DisableVolumeReplication
added early check for disabling and disabled
image mirroring state in DisableVolumeReplication

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-04-05 08:53:40 +00:00
Madhu Rajanna
fb3f7fe202 rbd: remove todo for image not found
Incase of resync the image will get deleted, gets
recreated and its a a time consuming operation.
It makes sense to return aborted error instead
of not found as we have omap data only the image
is missing in rbd pool.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-04-05 08:53:40 +00:00
Madhu Rajanna
95387c3b5e rbd: check for peer site status
Do resync if the image is in unknow or in error
state.

Check for the current image state for up+stopped
or up+replaying and also all peer site status
should be un up+stopped to confirm that resyncing
is done and image can be promoted and used.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-04-05 08:53:40 +00:00
Madhu Rajanna
233954bc10 rbd: make replication operations as rbdImage methods
added replication related operations as a method
of rbdImage as these methods can be easily used
when we introduce volumesnaphot mirroring operations.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-04-05 08:53:40 +00:00
Madhu Rajanna
c822ad460d rbd: add a check for image mirror disabling state
the rbd mirror state can be in enabled,disabled
or disabling state. If the mirroring is not disabled
yet and still in disabling state. we need to
check for it and return abort error message if
the mirroring is still getting disabled.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-04-05 08:53:40 +00:00
Madhu Rajanna
aaf6b571b8 rbd: Add ReplicationServer struct for replication operations
added ReplicationServer struct for the replication related
operation it also embed the ControllerServer which
already implements the helper functions like locking/unlocking etc.

removed getVolumeFromID and cleanup functions for better
code readability and easy maintaince.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-04-05 08:53:40 +00:00
Niels de Vos
eea97ca014 util: move GetID() from EncryptionKMS to VolumeEncryption
There is no need for each EncryptionKMS to implement the same GetID()
function. We have a VolumeEncryption type that is more suitable for
keeping track of the KMS-ID that was used to get the configuration of
the KMS.

This does not change any metadata that is stored anywhere.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-03-24 12:09:04 +00:00
Madhu Rajanna
d8f7b38d3d rbd: add exclusive-lock and journaling image features for rbd image
Current rbd plugin only supports the layering feature
for rbd image. Add exclusive-lock and journaling image
features for the rbd.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
Signed-off-by: woohhan <woohyung_han@tmax.co.kr>
2021-03-24 09:48:04 +00:00
Niels de Vos
bbd24e52f3 cleanup: use rbdImage.Destroy() for temporary volumes
rbdVolumes can have several resources that get allocated during its
usage. Only destroying the IOContext may not be suffiecient and can
cause resource leaks.

Use rbdVolume.Destroy() when the rbdVolume is not used anymore.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-03-17 07:50:09 +00:00
Niels de Vos
5c26fbb0d7 util: use ClusterConnection.Copy() for re-using connections
Connections are reference counted, so just assigning the connection to
an other object for re-use is not correct. This can cause connections to
be garbage collected while something else is still using it.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-03-17 07:50:09 +00:00
Madhu Rajanna
6e941539b5 rbd: implement volume replication spec
implemented the volume replication spec
for the rbd mirroring.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-03-16 13:06:44 +00:00
Madhu Rajanna
ee9a200fcc rbd: implement mirroring helpers with go-ceph
mirror.go exposes the helper functions to
perform the mirroring operations.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-03-16 13:06:44 +00:00
Madhu Rajanna
c642637cec util: register replication controller
as RBD is implementing the replication
we are registering it. For CephFS, its
not implementing the replication we are
passing nil so we dont want to register
it.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-03-16 13:06:44 +00:00
Madhu Rajanna
ee0576278f cleanup: move servers to a new struct
For future enhancments like adding more
servers. Moving the list of servers to a
new structure.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-03-16 13:06:44 +00:00
Niels de Vos
10a75dd4ff rbd: introduce rbdImage as base for rbdVolume and rbdSnapshot
Because rbdVolume and rbdSnapshot are very similar, they can be based
off a common struct rbdImage that contains the common attributes and
functions.

This makes it possible to re-use functions for snapshots, and prevents
further duplication or code.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-03-15 08:10:51 +00:00
Niels de Vos
3fe714c4fa cleanup: rename rbdSnapshot.SnapID to VolID
The rbdSnapshot and rbdVolume structs have many common attributes. In
order to combine these into an rbdImage struct that implements shared
functionality, having the same attribute for the ID makes things much
easier.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-03-15 08:10:51 +00:00
Niels de Vos
cffec0b3f3 rbd: configure the DEKStore if the configuration suggests to use metadata
NewVolumeEncryption() will return an indication that an alternative
DEKStore needs to be configured in case the KMS does not support it.

setKMS() will also set the DEKStore if needed, so renaming it to
configureEncryption() makes things clearer.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-03-12 10:11:47 +00:00
Niels de Vos
e4431edaf9 rbd: implement the DEKStore interface
To accommodate storing DEKs outside a KMS, the DEK can be stored in the
metadata of the volume.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-03-12 10:11:47 +00:00
Niels de Vos
9ac7f56400 util: move existing KMS implementations to the DEKStore interface
Use DEKStore API for Fetching and Storing passphrases.

Drop the fallback for the old KMS interface that is now provided as
DEKStore. The original implementation has been re-used for the DEKStore
interface.

This also moves GetCryptoPassphrase/StoreNewCryptoPassphrase functions
to methods of VolumeEncryption.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-03-12 10:11:47 +00:00
Niels de Vos
d4076d6216 util: introduce VolumeEncryption type
Prepare for grouping encryption related functions together. The main
rbdVolume object should not be cluttered with KMS or DEK procedures.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-03-12 10:11:47 +00:00
Madhu Rajanna
cbb10fd84d rbd: add more logging for NodeUnstageVolume
For NodeUnstageVolume its a two step process,
first unmount the volume and than unmap the volume.
Currently, we are logging only after rbd unmapping is done.
sometimes it becomes difficult to debug with above logging
whether more time is spent in unmount or unmap.
This commits adds one more debug log after unmount is done.
with this we can identify where exactly more time is spent
by looking at the logs.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-03-11 17:40:57 +00:00
Niels de Vos
fe0f169875 rbd: write max 1gb per WriteSame() operation
It seems that writing more than 1 GiB per WriteSame() operation causes
an EINVAL (22) "Invalid argument" error. Splitting the writes in blocks
of maximum 1 GiB should prevent that from happening.

Not all volumes are of a size that is the multiple of the stripe-size.
WriteSame() needs to write full blocks of data, so in case there is a
small left-over, it will be filled with WriteAt().

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-03-11 10:57:31 +00:00
Niels de Vos
165a837bca rbd: move KMS initialization into rbdVol.initKMS()
Introduce initKMS() as a function of rbdVolume. KMS functionality does
not need to pollute general RBD image functions. Encryption functions
are now in internal/rbd.encryption.go, so move initKMS() there as well.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-02-24 13:16:11 +00:00
Niels de Vos
cf6dae86e9 rbd: move encryptDevice() to a method of rbdVolume
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-02-24 13:16:11 +00:00
Niels de Vos
fb065b0f39 rbd: move openEncryptedDevice() to a method of rbdVolume
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-02-24 13:16:11 +00:00
Niels de Vos
b5020657e6 rbd: add "--options notrim" when mapping a thick-provisioned image
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-02-19 11:55:40 +00:00
Niels de Vos
cc96bdaac3 rbd: allocate extents when expanding an image
When and RBD image is expanded, the additional extents need to get
allocated when the image was thick-provisioned.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-02-19 11:55:40 +00:00
Niels de Vos
294a0973bd rbd: mark images thick-provisioned in metadata
When images get resized/expanded, the additional space needs to be
allocated if the image was initially thick-provisioned. By marking the
image with a "thick-provisioned" key in the metadata, future operations
can check the need.

A missing "thick-provisioned" key indicates that the image has not been
thick-provisioned.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-02-19 11:55:40 +00:00
Niels de Vos
74d218df8d rbd: disable rbd_discard_on_zeroed_write_same for thick-allocation
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-02-19 11:55:40 +00:00
Niels de Vos
5522a05f59 rbd: thick-provision images on request
Write blocks of stripe-size to allocate RBD images when
Thick-Provisioning is enabled in the StorageClass.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-02-19 11:55:40 +00:00
Madhu Rajanna
c417a5d0ba rbd: add support for thick provisioning option
Add an option to the StorageClass to support creating fully allocated
(thick provisioned) RBD images

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-02-19 11:55:40 +00:00
Niels de Vos
4937e59c4d rbd: add backwards compatible encryption in NodeStageVolume
When a volume was provisioned by an old Ceph-CSI provisioner, the
metadata of the RBD image will contain `requiresEncryption` to indicate
a passphrase needs to be created. New Ceph-CSI provisioners create the
passphrase in the CreateVolume request, and set `encryptionPrepared`
instead.

When a new node-plugin detects that `requiresEncryption` is set in the
RBD image metadata, it will fallback to the old behaviour.

In case `encryptionPrepared` is read from the RBD image metadata, the
passphrase is used to cryptsetup/format the image.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-02-17 17:51:13 +00:00
Niels de Vos
ee79b22c97 rbd: move encryption function to encryption.go
This adds internal/rbd/encryption.go which will be used to include other
encryption functionality to support additional KMS related functions. It
will work together with the shared API from internal/util/kms.go.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-02-17 17:51:13 +00:00