Prepared for an enhanced API to communicate with a KMS and keep the DEK
storage separate. The crypto.go file is already mixed with different
functions, so moving the KMS part into its own file, just like we have
for Hashicorp Vault KMS's.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
For NodeUnstageVolume its a two step process,
first unmount the volume and than unmap the volume.
Currently, we are logging only after rbd unmapping is done.
sometimes it becomes difficult to debug with above logging
whether more time is spent in unmount or unmap.
This commits adds one more debug log after unmount is done.
with this we can identify where exactly more time is spent
by looking at the logs.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
The stripe-size is the most efficient size to write to RBD images.
However, not all images are a multiple of stripe-size large. That means
thick-provisioning would not allocate the full image, and the process
might even fail.
This adds a 50 MB PVC to test the process, 100 MB is coincidentally a
multiple of the (default 4 MB) stripe-size, 50 MB is not.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
It seems that writing more than 1 GiB per WriteSame() operation causes
an EINVAL (22) "Invalid argument" error. Splitting the writes in blocks
of maximum 1 GiB should prevent that from happening.
Not all volumes are of a size that is the multiple of the stripe-size.
WriteSame() needs to write full blocks of data, so in case there is a
small left-over, it will be filled with WriteAt().
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Minikube v1.18 has been released. This conains a fix for our `losetup`
workaround, so that can be removed now.
Updates: #1840
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Mergify does now support a `defaults:` section. It can be used to
configure the `bot_account:` in a single location.
CC: @ceph-csi-bot
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Introduce initKMS() as a function of rbdVolume. KMS functionality does
not need to pollute general RBD image functions. Encryption functions
are now in internal/rbd.encryption.go, so move initKMS() there as well.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
when user provides an option for VolumeNamePrefix
create subvolume with the prefix which will be easy
for user to identify the subvolumes belongs to
the storageclass, Added an E2E testing to verify
that the subvolume contains the Prefix what is
provided in the storageclass.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
when user provides an option for VolumeNamePrefix
create subvolume with the prefix which will be easy
for user to identify the subvolumes belongs to
the storageclass.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
set system-cluster-critical priorityclass on
provisioner pods. the system-cluster-critical is
having lowest priority compared to node-critical.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
set system-node-critical priority on the plugin
pods, as its the highest priority and this need to
be applied on plugin pods as its critical for
storage in cluster.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
When one Quantity is in GiB, and the other in Dec (bytes), the value
should be the same. However, by using ==, this is not the case. It is
needed to use Equals() for that.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
When and RBD image is expanded, the additional extents need to get
allocated when the image was thick-provisioned.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
When images get resized/expanded, the additional space needs to be
allocated if the image was initially thick-provisioned. By marking the
image with a "thick-provisioned" key in the metadata, future operations
can check the need.
A missing "thick-provisioned" key indicates that the image has not been
thick-provisioned.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Write blocks of stripe-size to allocate RBD images when
Thick-Provisioning is enabled in the StorageClass.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Add an option to the StorageClass to support creating fully allocated
(thick provisioned) RBD images
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
Signed-off-by: Niels de Vos <ndevos@redhat.com>
When deleting a PVC fails, the following messages are repeated until a
timeout is hit:
cephfs-80811 in state &PersistentVolumeClaimStatus{Phase:,AccessModes:[],Capacity:ResourceList{},Conditions:[]PersistentVolumeClaimCondition{},} to be deleted (600 seconds elapsed)
Because the Phase is not set, the PVC seems to be in a strange state. In
case this happens, log all details from the PVC so that we can identify
additional conditions to check for completed deletion.
Updates: #1874
Signed-off-by: Niels de Vos <ndevos@redhat.com>
When a volume was provisioned by an old Ceph-CSI provisioner, the
metadata of the RBD image will contain `requiresEncryption` to indicate
a passphrase needs to be created. New Ceph-CSI provisioners create the
passphrase in the CreateVolume request, and set `encryptionPrepared`
instead.
When a new node-plugin detects that `requiresEncryption` is set in the
RBD image metadata, it will fallback to the old behaviour.
In case `encryptionPrepared` is read from the RBD image metadata, the
passphrase is used to cryptsetup/format the image.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
This adds internal/rbd/encryption.go which will be used to include other
encryption functionality to support additional KMS related functions. It
will work together with the shared API from internal/util/kms.go.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Storing a passphrase is now done while the volume is created. There is
no need to (re)generate a passphrase when it can not be found.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Have the provisioner create the passphrase for the volume, instead of
doign it lazily at the time the volume is used for the 1st time. This
prevents potential races where pods on different nodes try to store
different passphrases at the (almost) same time.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
When passing a pointer to a PVC and PV, the status of the deleted
objects is not logged correctly. The `PersistentVolumeClaim.Status` and
`PersistedVolume.Status` that is added to the logs contain the status of
the initially created object (reference to the PVC/PV). When the PVC/PV
is removed, there is no guarantee that the object is updated.
Logs show an empty (nullified) `PersistentVolumeClaim.Status`, which is
not helpful. Instead, use the returned PVC/PV from the `Get()` function
and use that for further logging. Even when the `.Status` struct from
the PVC/PV gets wiped, the returned object should have correct details.
Updates: #1874
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Failures when deploying Hashicorp Vault are logged as informative. This
means that testing will continue, even if Vault will not be available.
Instead of logging the errors as INFO, use FAIL so that tests are not
run and the problems are identified early and obviously.
Updates: #1795
Signed-off-by: Niels de Vos <ndevos@redhat.com>
"VAULT_SKIP_VERIFY" is a standard Hashicorp Vault environment variable
(a string) that needs to get converted to the "vaultCAVerify"
configuration option in the Ceph-CSI format.
The value of "VAULT_SKIP_VERIFY" means the reverse of "vaultCAVerify",
this part was missing in the original conversion too.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
Changes:
1. Add a variable in build.env for rook ceph cluster version.
2. Modify rook.sh so that it can deploy ceph cluster with
desirable version also rather than the one which rook installs
by default.
3. Remove the code which is no longer required:
a. Code which was added to test snapshot feature.
b. Code which was required because
https://github.com/rook/rook/pull/5925 was not fixed.
Signed-off-by: Mudit Agarwal <muagarwa@redhat.com>
It seems that calls to addRbdManagerTask() do not include the
rados-namespace in the image location. Functions calling
addRbdManagerTask() construct the image location themselves, but should
use rbdVolume.String() to include all the attributes.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
The e2e tests create a Secret for using with the RBD StorageClass.
However this Secret was not used, instead the Rook generated Secret was
linked in the StorageClass.
By using our own Secret from the examples, Rook should not touch it when
we make modifications. In addition, no modifications are needed for
encryption anymore, as these are included in the example.
Updates: #1795
Signed-off-by: Niels de Vos <ndevos@redhat.com>
There are many usecases with adding the subvolume
path to the PV object. the volume context returned
in the createVolumeResponse is added to the PV object
by the external provisioner.
More Details about the usecases are in below link
https://github.com/rook/rook/issues/5471
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
Sometimes testing takes more than 60 minutes. When that is the case, the
60 minute timeout causes a golang panic in the test suite.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
It seems that recent minikube versions changed something in the
networking, and that prevents
$ ceph fs subvolumegroup create myfs testGroup
from working. Strangely RBD is not impacted. Possibly something is
confusing the CephMgr pod that handles the CephFS admin commands.
Using the "bridge" CNI seems to help, CephFS admin commands work with
this in minikube.
Signed-off-by: Niels de Vos <ndevos@redhat.com>
