Commit Graph

689 Commits

Author SHA1 Message Date
Madhu Rajanna
d8f7b38d3d rbd: add exclusive-lock and journaling image features for rbd image
Current rbd plugin only supports the layering feature
for rbd image. Add exclusive-lock and journaling image
features for the rbd.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
Signed-off-by: woohhan <woohyung_han@tmax.co.kr>
2021-03-24 09:48:04 +00:00
Yati Padia
0d9548c815 Cephfs: Failed to delete snapshot
Failed to delete voluesnapshot when backend subvolume
(pvc) and ceph fs subvolume snapshot is deleted

Fixes#1647

Signed-off-by: Yati Padia <ypadia@redhat.com>
2021-03-17 10:28:08 +00:00
Niels de Vos
bbd24e52f3 cleanup: use rbdImage.Destroy() for temporary volumes
rbdVolumes can have several resources that get allocated during its
usage. Only destroying the IOContext may not be suffiecient and can
cause resource leaks.

Use rbdVolume.Destroy() when the rbdVolume is not used anymore.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-03-17 07:50:09 +00:00
Niels de Vos
5c26fbb0d7 util: use ClusterConnection.Copy() for re-using connections
Connections are reference counted, so just assigning the connection to
an other object for re-use is not correct. This can cause connections to
be garbage collected while something else is still using it.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-03-17 07:50:09 +00:00
Madhu Rajanna
6e941539b5 rbd: implement volume replication spec
implemented the volume replication spec
for the rbd mirroring.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-03-16 13:06:44 +00:00
Madhu Rajanna
ee9a200fcc rbd: implement mirroring helpers with go-ceph
mirror.go exposes the helper functions to
perform the mirroring operations.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-03-16 13:06:44 +00:00
Madhu Rajanna
c642637cec util: register replication controller
as RBD is implementing the replication
we are registering it. For CephFS, its
not implementing the replication we are
passing nil so we dont want to register
it.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-03-16 13:06:44 +00:00
Madhu Rajanna
ee0576278f cleanup: move servers to a new struct
For future enhancments like adding more
servers. Moving the list of servers to a
new structure.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-03-16 13:06:44 +00:00
Niels de Vos
10a75dd4ff rbd: introduce rbdImage as base for rbdVolume and rbdSnapshot
Because rbdVolume and rbdSnapshot are very similar, they can be based
off a common struct rbdImage that contains the common attributes and
functions.

This makes it possible to re-use functions for snapshots, and prevents
further duplication or code.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-03-15 08:10:51 +00:00
Niels de Vos
3fe714c4fa cleanup: rename rbdSnapshot.SnapID to VolID
The rbdSnapshot and rbdVolume structs have many common attributes. In
order to combine these into an rbdImage struct that implements shared
functionality, having the same attribute for the ID makes things much
easier.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-03-15 08:10:51 +00:00
Niels de Vos
5e63743243 util: add SecretsMetadataKMS
This new KMS is based on the (default) SecretsKMS, but instead of using
the passphrase for all volumes, the passphrase is used to
encrypt/decrypt a Data-Encryption-Key that is stored in the metadata of
the volume.

CC: Patrick Uiterwijk <puiterwijk@redhat.com> - for encryption guidance
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-03-12 10:11:47 +00:00
Niels de Vos
6915624380 util: add EncryptDEK DecryptDEK to EncryptionKMS interface
By adding these methods, a KMS can explicitly encrypt/decrypt the DEK if
there is no transparent way of doing so.

Hashicorp Vault encrypts the DEK when it it stored, and decrypts it when
fetched. Therefor there is no need to do any encryption in this case.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-03-12 10:11:47 +00:00
Niels de Vos
cffec0b3f3 rbd: configure the DEKStore if the configuration suggests to use metadata
NewVolumeEncryption() will return an indication that an alternative
DEKStore needs to be configured in case the KMS does not support it.

setKMS() will also set the DEKStore if needed, so renaming it to
configureEncryption() makes things clearer.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-03-12 10:11:47 +00:00
Niels de Vos
e4431edaf9 rbd: implement the DEKStore interface
To accommodate storing DEKs outside a KMS, the DEK can be stored in the
metadata of the volume.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-03-12 10:11:47 +00:00
Niels de Vos
9ac7f56400 util: move existing KMS implementations to the DEKStore interface
Use DEKStore API for Fetching and Storing passphrases.

Drop the fallback for the old KMS interface that is now provided as
DEKStore. The original implementation has been re-used for the DEKStore
interface.

This also moves GetCryptoPassphrase/StoreNewCryptoPassphrase functions
to methods of VolumeEncryption.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-03-12 10:11:47 +00:00
Niels de Vos
b60dd286c6 util: use the KMS as DEKStore if it supports it
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-03-12 10:11:47 +00:00
Niels de Vos
ee033da8e9 util: add DEKStore interface
DEKStore is a new interface that will be used for Storing and Fetching
DEKs. The existing implementations for KMS already function as a
DEKStore, and will be updated to match the interface.

By splitting KMS and DEKStore into two components, the encryption
configuration for volumes becomes more modular. This makes it possible
to implement a DEKStore where the encrypted DEK for a volume is stored
in the metadata of the volume (RBD image).

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-03-12 10:11:47 +00:00
Niels de Vos
d4076d6216 util: introduce VolumeEncryption type
Prepare for grouping encryption related functions together. The main
rbdVolume object should not be cluttered with KMS or DEK procedures.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-03-12 10:11:47 +00:00
Niels de Vos
aa52afff09 cleanup: move SecretsKMS in own file
Prepared for an enhanced API to communicate with a KMS and keep the DEK
storage separate. The crypto.go file is already mixed with different
functions, so moving the KMS part into its own file, just like we have
for Hashicorp Vault KMS's.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-03-12 10:11:47 +00:00
Madhu Rajanna
cbb10fd84d rbd: add more logging for NodeUnstageVolume
For NodeUnstageVolume its a two step process,
first unmount the volume and than unmap the volume.
Currently, we are logging only after rbd unmapping is done.
sometimes it becomes difficult to debug with above logging
whether more time is spent in unmount or unmap.
This commits adds one more debug log after unmount is done.
with this we can identify where exactly more time is spent
by looking at the logs.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-03-11 17:40:57 +00:00
Niels de Vos
fe0f169875 rbd: write max 1gb per WriteSame() operation
It seems that writing more than 1 GiB per WriteSame() operation causes
an EINVAL (22) "Invalid argument" error. Splitting the writes in blocks
of maximum 1 GiB should prevent that from happening.

Not all volumes are of a size that is the multiple of the stripe-size.
WriteSame() needs to write full blocks of data, so in case there is a
small left-over, it will be filled with WriteAt().

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-03-11 10:57:31 +00:00
Niels de Vos
6c8bc79771 ci: add unit tests for SecretsKMS
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-03-04 12:11:07 +00:00
Niels de Vos
165a837bca rbd: move KMS initialization into rbdVol.initKMS()
Introduce initKMS() as a function of rbdVolume. KMS functionality does
not need to pollute general RBD image functions. Encryption functions
are now in internal/rbd.encryption.go, so move initKMS() there as well.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-02-24 13:16:11 +00:00
Niels de Vos
cf6dae86e9 rbd: move encryptDevice() to a method of rbdVolume
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-02-24 13:16:11 +00:00
Niels de Vos
fb065b0f39 rbd: move openEncryptedDevice() to a method of rbdVolume
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-02-24 13:16:11 +00:00
Madhu Rajanna
8720f4e2f5 cephfs: create subvolume with VolumeNamePrefix
when user provides an option for VolumeNamePrefix
create subvolume with the prefix which will be easy
for user to identify the subvolumes belongs to
the storageclass.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-02-19 17:04:17 +00:00
Niels de Vos
b5020657e6 rbd: add "--options notrim" when mapping a thick-provisioned image
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-02-19 11:55:40 +00:00
Niels de Vos
cc96bdaac3 rbd: allocate extents when expanding an image
When and RBD image is expanded, the additional extents need to get
allocated when the image was thick-provisioned.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-02-19 11:55:40 +00:00
Niels de Vos
294a0973bd rbd: mark images thick-provisioned in metadata
When images get resized/expanded, the additional space needs to be
allocated if the image was initially thick-provisioned. By marking the
image with a "thick-provisioned" key in the metadata, future operations
can check the need.

A missing "thick-provisioned" key indicates that the image has not been
thick-provisioned.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-02-19 11:55:40 +00:00
Niels de Vos
74d218df8d rbd: disable rbd_discard_on_zeroed_write_same for thick-allocation
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-02-19 11:55:40 +00:00
Niels de Vos
5522a05f59 rbd: thick-provision images on request
Write blocks of stripe-size to allocate RBD images when
Thick-Provisioning is enabled in the StorageClass.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-02-19 11:55:40 +00:00
Madhu Rajanna
c417a5d0ba rbd: add support for thick provisioning option
Add an option to the StorageClass to support creating fully allocated
(thick provisioned) RBD images

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-02-19 11:55:40 +00:00
Niels de Vos
4937e59c4d rbd: add backwards compatible encryption in NodeStageVolume
When a volume was provisioned by an old Ceph-CSI provisioner, the
metadata of the RBD image will contain `requiresEncryption` to indicate
a passphrase needs to be created. New Ceph-CSI provisioners create the
passphrase in the CreateVolume request, and set `encryptionPrepared`
instead.

When a new node-plugin detects that `requiresEncryption` is set in the
RBD image metadata, it will fallback to the old behaviour.

In case `encryptionPrepared` is read from the RBD image metadata, the
passphrase is used to cryptsetup/format the image.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-02-17 17:51:13 +00:00
Niels de Vos
ee79b22c97 rbd: move encryption function to encryption.go
This adds internal/rbd/encryption.go which will be used to include other
encryption functionality to support additional KMS related functions. It
will work together with the shared API from internal/util/kms.go.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-02-17 17:51:13 +00:00
Niels de Vos
dc81e001cf cleanup: remove unused MissingPassphrase error type
Storing a passphrase is now done while the volume is created. There is
no need to (re)generate a passphrase when it can not be found.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-02-17 17:51:13 +00:00
Niels de Vos
9b6c2117f3 rbd: set encryption passphrase on CreateVolume
Have the provisioner create the passphrase for the volume, instead of
doign it lazily at the time the volume is used for the 1st time. This
prevents potential races where pods on different nodes try to store
different passphrases at the (almost) same time.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-02-17 17:51:13 +00:00
Niels de Vos
a42c4b5855 util: convert VAULT_SKIP_VERIFY to "vaultCAVerify" KMS option
"VAULT_SKIP_VERIFY" is a standard Hashicorp Vault environment variable
(a string) that needs to get converted to the "vaultCAVerify"
configuration option in the Ceph-CSI format.

The value of "VAULT_SKIP_VERIFY" means the reverse of "vaultCAVerify",
this part was missing in the original conversion too.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-02-16 13:05:47 +00:00
Niels de Vos
d534ee9ce8 rbd: include rados-namespace when calling addRbdManagerTask()
It seems that calls to addRbdManagerTask() do not include the
rados-namespace in the image location. Functions calling
addRbdManagerTask() construct the image location themselves, but should
use rbdVolume.String() to include all the attributes.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-02-12 12:02:14 +00:00
Niels de Vos
8d0b39e690 rbd: log error when scheduling flattening fails
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-02-12 12:02:14 +00:00
Madhu Rajanna
8cd901d2dd cephfs: add subvolume path to volume context
There are many usecases with adding the subvolume
path to the PV object. the volume context returned
in the createVolumeResponse is added to the PV object
by the external provisioner.

More Details about the usecases are in below link
https://github.com/rook/rook/issues/5471

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-02-11 11:31:22 +00:00
Madhu Rajanna
dd6ce7b441 rbd: fix error check when reading vaultCAFromSecret
check correct error variable when reading
vaultCAFromSecret.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-02-04 14:58:40 +00:00
Madhu Rajanna
e9782d86ad rbd: fix incorrect reading of client cert key
fix incorrect reading of client cert key.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-02-04 14:58:40 +00:00
Madhu Rajanna
f63ccb0cce rbd: store VaultCAVerify as a string
storing VaultCAVerify as a string.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-02-04 14:58:40 +00:00
Madhu Rajanna
bf5c36822f rbd: set tenant in kms object
the tenant/namespace is needed to read the certificates,
this commit sets the tenant in kms object.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-02-04 14:58:40 +00:00
Madhu Rajanna
22ae4a0b16 rbd: change key in secret for cert and tls
currently, the keys for kms certificates/keys in a
secret is ca.cert, tls.cert and
tls.key, this commit changes the key from ca.cert
and tls.cert to cert and tls.key to key.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-02-04 14:58:40 +00:00
Madhu Rajanna
b370d9afb6 rbd: unmarshal the data read from file
if are reading the kms data from the file.
than only we need to unmarshal. If we are reading
from the configmap it already returns the unmarshal
data.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-02-04 14:58:40 +00:00
Niels de Vos
582d004ca5 util: store EnvVaultInsecure as string, not bool
The configuration option `EnvVaultInsecure` is expected to be a string,
not a boolean. By converting the bool back to a string (after
verification), it is now possible to skip the certificate validation
check by setting `vaultCAVerify: false` in the Vault configuration.

Fixes: #1852
Reported-by: Bryon Nevis <bryon.nevis@intel.com>
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-02-02 11:24:28 +00:00
Niels de Vos
df81022349 rbd: add support for VAULT_SKIP_VERIFY in KMS ConfigMap
When the KMS VaultTokens is configured through a Kubernetens ConfigMap,
the `VAULT_SKIP_VERIFY` option was not taken into account. The option
maps to the `vaultCAVerify` value in the configuration file, but has the
reverse meaning.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-02-01 13:09:54 +00:00
Mudit Agarwal
d480eb4bda cephfs: ignore BytesQuota field in case it is not set.
This can happen when the subvolume is in snapshot-retained state.
We should not return error for such case as it is a valid situation.

Signed-off-by: Mudit Agarwal <muagarwa@redhat.com>
2021-02-01 09:20:53 +00:00
Madhu Rajanna
584a43dc2c rbd: fix issue in ENV variable check
Currently cephcsi is returning an error
if the ENV variable is set, but it should not.
This commit fixes the the POD_NAMESPACE env
variable issue and as well as the KMS_CONFIG_NAME
ENV variable.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2021-01-29 10:00:12 +00:00
Niels de Vos
0b7521162c cleanup: rewrite ifElseChains to switch statements
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2021-01-27 13:03:56 +00:00
Ilya Dryomov
04644c1d58 rbd: enable mapping and unmapping from a network namespace
Make rbdplugin pod work in a non-initial network namespace (i.e. with
"hostNetwork: false") by skipping waiting for udev events when mapping
and unmapping images.  CSI use case is very simple: all that is needed
is a device node which is immediately fed to mkfs, so we should be able
to tolerate udev not being finished with the device just fine.

Fixes: #1323
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2021-01-07 15:34:05 +00:00
Ilya Dryomov
c2493686b7 rbd: introduce appendDeviceTypeAndOptions()
Factor out --device-type and --options formatting.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2021-01-07 15:34:05 +00:00
Ilya Dryomov
d3f31187fc rbd: rename ndbType parameter
Fix "ndb" typo.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2021-01-07 15:34:05 +00:00
Ilya Dryomov
5631b83dd0 rbd: rename mapOptions and options argument slices
With the new support for passing --options, referring to ExecCommand()
argument slices as mapOptions and options is confusing.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2021-01-07 15:34:05 +00:00
Seena Fallah
fdec9f65b8 rbd: fix namespace json parser for xbdDeviceInfo
rbd device list --format=json returns namespace as a namespace not radosNamespace

Signed-off-by: Seena Fallah <seenafallah@gmail.com>
2021-01-05 11:26:09 +00:00
Yati Padia
995879d349 cephfs: use enum to check resize is supported or not
Currently, we are using bool pointer to find out
the ceph cluster supports resize or not. This
commit replaces the bool pointer with enum.

Signed-off-by: Yati Padia <ypadia@redhat.com>

Fixes#1764
2021-01-04 04:42:58 +00:00
Madhu Rajanna
9c7176dbb4 rbd: update mount packges in import path
mount packges is moved from
k8s.io/utils/mount to a new repository
k8s.io/mount-utils. updated code to use
the same.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-17 16:04:54 +00:00
Madhu Rajanna
b3fbcb9c95 rbd: read configuration from the configmap
if the kms encryption configmap is not mounted
as a volume to the CSI pods, add the code to
read the configuration from the kubernetes. Later
the code to fetch the configmap will be moved to
the new sidecar which is will talk to respective
CO to fetch the encryption configurations.

The k8s configmap uses the standard vault spefic
names to add the configurations. this will be converted
back to the CSI configurations.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-16 15:05:05 +00:00
Niels de Vos
e4b16a5c72 util: allow tenants to (re)configure VaultTokens settings
A tenant can place a ConfigMap in their Kubernetes Namespace with
configuration options that differ from the global (by the Storage Admin
set) values.

The ConfigMap needs to be located in the Tenants namespace, as described
in the documentation

See-also: docs/design/proposals/encryption-with-vault-tokens.md
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-16 13:42:52 +00:00
Madhu Rajanna
81061e9f68 util: add support for vault certificates
Added a option to pass the client certificate
and the client certificate key for the vault token
based encryption.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-16 11:01:15 +00:00
Niels de Vos
db15458d16 cleanup: use constant for "vault" KMS-type
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-14 14:45:09 +00:00
Niels de Vos
cc5684dbd8 util: add support for Hashicorp Vault with Tokens per Tenant
Tenants (Kubernetes Namespaces) can use their own Vault Token to manage
the encryption keys for PVCs. The working is documented in #1743.

See-also: #1743
Closes: #1500
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-14 14:45:09 +00:00
Niels de Vos
648f9ccf31 util: support vaultBackendPath and vaultTLSServerName options
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-14 14:45:09 +00:00
Niels de Vos
8f91c672d4 util: add EncryptionKMS.Destroy()
Add a new method to the EncryptionKMS interface so that resources can be
freed when EncryptionKMS instances get freed.

With the move to using the libopenstorage API, a temporary file needs to
store the optional CA certificate. The Destroy() method of the
vaultConnection type now removes this file.

The rbdVolume uses the EncryptionKMS type now, so call the new Destroy()
method from withing rbdVolume.Destroy().

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-14 14:45:09 +00:00
Niels de Vos
eb1ef69cfb util: allow updating settings of vaultConnection
Make it possible to calle initConnection() multiple times. This enables
the VaultTokensKMS type to override global settings with options from a
per-tenant configuration.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-14 14:45:09 +00:00
Niels de Vos
f08182e2fc rbd: pass Owner to GetKMS()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-14 14:45:09 +00:00
Niels de Vos
5bbab25a9f util: move Secrets to vaultConnection
The Secrets is the main object to connect to Vault. This should be part
of the vaultConnection type.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-14 14:45:09 +00:00
Niels de Vos
d28a5a5f23 util: pass map[string]interface{} to initialize KMS
This makes it possible to pass a more complex configuration to the
initialize functions for KMS's. The upcoming VaultTokensKMS can use
overrides for configiration options on a per tenant basis. Without this
change, it would not be possible to consume the JSON configuration file.

See-also: #1743
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-14 14:45:09 +00:00
Niels de Vos
43fa1cddb7 util: use helper function to parse Vault configuration
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-14 14:45:09 +00:00
Madhu Rajanna
0d9fcbc21b cephfs: remove unused cr Credentials
removed unused cr variable.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-10 10:13:55 +00:00
Madhu Rajanna
9a96370942 cephfs: implement purgeVolume with go-ceph
moved frm  ceph fs CLI to go-ceph for
purgeVolume.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-10 10:13:55 +00:00
Madhu Rajanna
34d0ff0d70 cephfs: make purgeVolume method of volumeOptions
converted purgeVolume from function to method
of volumeOptions.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-10 10:13:55 +00:00
Niels de Vos
16cb43f0f9 rbd: store csi.storage.k8s.io/pvc/namespace metadata as Owner
The Owner of an RBD image (Kubernetes Namespace, tenant) can be used to
identify additional configuration options. This will be used for
fetching the right Vault Token when encrypting/decrypting volumes.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-09 13:58:48 +00:00
Niels de Vos
9160a5309e cleanup: standardize error format in VaultKMS.GetPassphrase()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-09 08:35:35 +00:00
Niels de Vos
e3ea8ca0b1 cleanup: standardize error format in util.GetTopologyFromDomainLabels()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-09 08:35:35 +00:00
Niels de Vos
6d5de7458d cleanup: standardize error format in util.k8sGetNodeLabels()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-09 08:35:35 +00:00
Niels de Vos
752841d213 cleanup: standardize error format in util.readClusterInfo()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-09 08:35:35 +00:00
Niels de Vos
3dfe65d17c cleanup: return error type in GetCryptoPassphrase()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-09 08:35:35 +00:00
Niels de Vos
c8c8176a37 cleanup: return error type in util.storeKey()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-09 08:35:35 +00:00
Niels de Vos
f8ebc6aa3f cleanup: return error type in ensureEncryptionMetadataSet()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-09 08:35:35 +00:00
Niels de Vos
d8e443ab49 cleanup: return error type in cleanupRBDImageMetadataStash()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-09 08:35:35 +00:00
Niels de Vos
f262673b60 cleanup: return error type in lookupRBDImageMetadataStash()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-09 08:35:35 +00:00
Niels de Vos
8e589587ae cleanup: return error type in stashRBDImageMetadata()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-09 08:35:35 +00:00
Niels de Vos
57ce07f54e cleanup: return error type in updateVolWithImageInfo()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-09 08:35:35 +00:00
Niels de Vos
610162b5f4 cleanup: return error type in genVolFromVolumeOptions()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-09 08:35:35 +00:00
Niels de Vos
729e2419ef cleanup: return error type in detachRBDImageOrDeviceSpec()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-09 08:35:35 +00:00
Niels de Vos
7eae69f10c cleanup: return error type in rbdGetDeviceList()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-09 08:35:35 +00:00
Niels de Vos
4dde3fc9e0 cleanup: return error type in encryptDevice()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-09 08:35:35 +00:00
Niels de Vos
d6fb8f302d cleanup: return error type in NodeServer.processEncryptedDevice()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-09 08:35:35 +00:00
Niels de Vos
8019e4d1bc rbd: return CSI status-error on resize failure
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-09 08:35:35 +00:00
Niels de Vos
65a10fd553 cleanup: standardize error format in NodeServer.NodeStageVolume()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-09 08:35:35 +00:00
Niels de Vos
cc3f146ad1 cleanup: return error type in rbdVolume.checkCloneImage()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-09 08:35:35 +00:00
Niels de Vos
37471c7a5f cleanup: return error type in ReconcilePersistentVolume.getCredentials()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-09 08:35:35 +00:00
Niels de Vos
a7a928d7ec cleanup: return error from execCommandErr() in bindMount()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-09 08:35:35 +00:00
Madhu Rajanna
43fde0a30a cleanup: add a helper function storeImageID
added a helper function storeImageID to reduce
code duplication.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-07 11:03:27 +00:00
Madhu Rajanna
b2fb43b335 cleanup: reduce the code complexity of controller
created a new helper function to getCredentials.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-07 11:03:27 +00:00
Madhu Rajanna
e243c0006b rbd: dont generate OMAP data for static volume
if the user has created a static PV for a RBD
image which is not created by CSI driver, dont
generate the OMAP data.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-07 11:03:27 +00:00
Madhu Rajanna
c40872df00 rbd: undo reservation incase of errors
If cephcsi encounters any error after
reservation, as a cleanup operation
it should revert back the reservation.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-07 11:03:27 +00:00
Madhu Rajanna
99dbe27921 rbd: return nil if the omap data exists
If the omap data already exits return nil.
so that omap generator will not try to reserve
anything again.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-07 11:03:27 +00:00
Madhu Rajanna
ebb413534f cephfs: remove unsed cr util.Credentials
remove unused cr util.Credentials variable.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-03 14:53:33 +00:00
Madhu Rajanna
0f451ed465 cephfs: implement getSnapshotInfo with go-ceph
implement getSnapshotInfo function with go-ceph
to get subvolume information.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-03 14:53:33 +00:00
Madhu Rajanna
ddf91de859 cephfs: implement clonesnapshot with go-ceph
updated ceph fs CLI implementaion of cloning
with go-ceph.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-03 06:39:18 +00:00
Madhu Rajanna
814bf4459a cephfs: implement snapshot protect and unprotect to go-ceph
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-03 06:39:18 +00:00
Niels de Vos
3433cefaa5 cleanup: standardize logging in deleteSnapshot
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-02 14:36:39 +00:00
Niels de Vos
294f7b22d4 cephfs: remove unused Credentials argument from deleteSnapshot()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-02 14:36:39 +00:00
Niels de Vos
9d9b5b3303 cephfs: implement deleteSnapshot() with go-ceph
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-02 14:36:39 +00:00
Niels de Vos
032db78d8a cephfs: make deleteSnapshot() a method of volumeOptions
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-02 14:36:39 +00:00
Niels de Vos
5883f244d2 cleanup: standardize logging in createSnapshot
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-02 13:18:47 +05:30
Niels de Vos
7c580af412 cephfs: remove unused Credentials argument from createSnapshot()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-02 13:18:47 +05:30
Niels de Vos
1d8cc75282 cephfs: implement createSnapshot() with go-ceph
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-02 13:18:47 +05:30
Niels de Vos
57f8ee0aa7 cephfs: make createSnapshot() a method of volumeOptions
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-12-02 13:18:47 +05:30
Madhu Rajanna
a35a835e9c cephfs: fix logic to call ceph fs resize
clusterAdditionalInfo map is holding a localClusterState
for checking ceph cluster supports resize and subvolumegroup
is created or not, currently we are checking if the key is present
in a map and localClusterStatelocalClusterState.resizeSupported
is set to false to call ceph fs subvolume resize to check command is
supported or not, if a structure is  initialized all its members
are set to default value. so we will never going to check the
ceph fs subvolume  resize command is supported in backend or not, we are
always using ceph fs subvolume create to resize subvolume. in some
ceph version ceph fs subvolume create wont work to resize a subvolume.
This commit changes the resizeSupported from bool to *bool for
proper handling of this scenario.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-01 13:31:39 +00:00
Madhu Rajanna
a1c8942e2b cephfs: remove unused invalidCommand error const
removed unused invalidCommand from cephfs
error const.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-01 09:46:52 +00:00
Madhu Rajanna
1ba160afdb cephfs: handle invalid error in subvolume info
go ceph returns NotImplementedError for invalid
commands,cephcsi is using errors.As to find out
the error.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-01 09:46:52 +00:00
Madhu Rajanna
66eb3b7525 cephfs: handle invalid error in fs resize
go ceph returns NotImplementedError for invalid
commands,cephcsi is using errors.As to find out
the error.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-01 09:46:52 +00:00
Madhu Rajanna
2232067e99 cephfs: use rados.ErrNotFound in subvolume info
go-ceph rados.ErrNotFound for not found errors,cephcsi
need to check same for not found errors.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-01 09:46:52 +00:00
Madhu Rajanna
a0283ef7f9 cephfs: use rados.ErrNotFound in getVolumeRootPathCeph
go-ceph rados.ErrNotFound for not found errors,cephcsi
need to check same for not found errors.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-01 09:46:52 +00:00
Madhu Rajanna
b40d561228 cephfs: make getVolumeRootPathCeph as a method
getVolumeRootPathCeph is a method of volumeOptions

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-01 09:46:52 +00:00
Madhu Rajanna
e4713bc9ed cephfs: remove unused Credentials argument from getVolumeRootPathCeph
remove unused Credentials argument from getVolumeRootPathCeph
function.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-01 09:46:52 +00:00
Madhu Rajanna
b6f3ba885e cephfs: implement getVolumeRootPathCeph with go-ceph
instead of ceph fs CLI commands using go-ceph library
to get subvolume path.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-12-01 09:46:52 +00:00
Niels de Vos
5565a902ac rebase: split vaultConnection from VaultKMS
In order to re-use the configuration of Vault, split a new
vaultConnection type from the VaultKMS type.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-30 19:16:39 +00:00
Madhu Rajanna
8ebb9a1ba0 cleanup: fix misspell words
fixed misspell words detected by  codespell

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-11-30 08:46:48 +01:00
Madhu Rajanna
39b1f2b4d3 cleanup: fix mispell words
fixed mispell words in the repo.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-11-29 12:47:46 +05:30
Niels de Vos
8d08e8b37e cephfs: remove unused Credentials argument from getCloneState()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-29 07:13:35 +00:00
Niels de Vos
055975f790 cephfs: implement getCloneState() with go-ceph
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-29 07:13:35 +00:00
Niels de Vos
476ad18d2e cephfs: make getCloneState() a method of volumeOptions
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-29 07:13:35 +00:00
Niels de Vos
66aa595e77 cephfs: do not export internal CloneStatus type
getCloneInfo() does not need to return a full CloneStatus struct that
only has one member. Instead, it can just return the value of the single
member, so the JSON type/struct does not need to be exposed.

This makes the API for getCloneInfo() a little simpler, so it can be
replaced by a go-ceph implementation later on.

As the function does not return any of the unused attributes anymore, it
is renamed to getCloneStatu() as well.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-29 07:13:35 +00:00
Niels de Vos
7824cb5ed7 util: use %w when forwarding an error
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-29 04:03:59 +00:00
Niels de Vos
5fba89f783 cleanup: use libopenstorage/secrets for Vault access
Instead of the hand-rolled Vault usage, use the libopenstorage/secrets
package that provides a nice API. The support for Vault becomes much
simpler and maintainable that way.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-29 04:03:59 +00:00
Madhu Rajanna
6091490393 rbd: improve logging in getCloneDepth
earlier if the depth check fails the
complete vol struct was getting logged,
this commits logs only the pool and image
name.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-11-28 18:50:00 +00:00
Madhu Rajanna
b3120926b9 rbd: remove extra Destory of parent volume
removed extra Destory of the parent volume.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-11-28 18:50:00 +00:00
Madhu Rajanna
30af703a2f rbd: add controller to main
initialize and start the rbd controller when
we the driver type is controller.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-11-28 18:50:00 +00:00
Madhu Rajanna
68bd44beba rbd: add new controller to regenerate omap data
In the case of Disaster Recovery failover, the
user expected to create the static PVC's. We have
planned not to go with the PVC name and namespace
for many reasons (as in kubernetes it's planned to
support PVC transfer to a new namespace with a
different name and with new features coming in
like data populator etc). For now, we are
planning to go with static PVC's to support
async mirroring.

During Async mirroring only the RBD images are
mirrored to the secondary site, and when the
user creates the static PVC's on the failover
we need to regenerate the omap data. The
volumeHandler in PV spec is an encoded string
which contains clusterID and poolID and image UUID,
The clusterID and poolID won't remain same on both
the clusters, for that cephcsi need to generate the
new volume handler and its to create a mapping
between new volume handler and old volume handler
with that whenever cephcsi gets csi requests it
check if the mapping exists it will pull the new
volume handler and continues other operations.

The new controller watches for the PVs created,
It checks if the omap exists if it doesn't it
will regenerate the entire omap data.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-11-28 18:50:00 +00:00
Madhu Rajanna
14700b89d1 rbd: update inuse logic of a rbd image
in case of mirrored image, if the image is
primary a watcher will be added by the rbd
mirror deamon on the rbd image.
we have to consider 2 watcher to check image
is in use.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-11-28 18:50:00 +00:00
Madhu Rajanna
ba84f14241 journal: create object with provided UUID
incase of async mirroring the volume UUID is
retrieved from the volume name, instead of cephcsi
generating a new UUID it should reserve the passed
UUID it will be useful when we support both metro DR
and async mirroring on a kubernetes clusters.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-11-28 18:50:00 +00:00
Madhu Rajanna
8d3a44d0c4 rbd: add minsnapshotsonimage flag
An rbd image can have a maximum number of
snapshots defined by maxsnapshotsonimage
On the limit is reached the cephcsi will
start flattening the older snapshots and
returns the ABORT error message, The Request
comes after this as to wait till all the
images are flattened (this will increase the
PVC creation time.  Instead of waiting till
the maximum snapshots on an RBD image, we can
have a soft limit, once the limit reached
cephcsi will start flattening the task to
break the chain. With this PVC  creation time
will only be affected when the hard limit
(minsnapshotsonimage) reached.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-11-18 05:59:20 +00:00
Yug
3ac6bbd87c cephfs: Add isCloneRetryError function
The function isCloneRetryError verifies
if the clone error is `pending` or
`in-progress` error.

Co-authored-by: Madhu Rajanna <madhupr007@gmail.com>
Signed-off-by: Yug <yuggupta27@gmail.com>
2020-11-09 07:29:12 +00:00
Yug
acbedc52bf cephfs: Add 'pending' state for clone status
In certain cases, clone status can be 'pending'.
In that case, abort error message should be
returned similar to that during 'in-progress'
state.

Co-authored-by: Madhu Rajanna <madhupr007@gmail.com>
Signed-off-by: Yug <yuggupta27@gmail.com>
2020-11-09 07:29:12 +00:00
Niels de Vos
565038fdfd cephfs: ignore quota when SubVolumeInfo() returns Infinite
There is a type-check on BytesQuota after calling SubVolumeInfo() to see
if the value is supported. In case no quota is configured, the value
Infinite is returned. This can not be converted to an int64, so the
original code returned an error.

It seems that attaching/mounting sometimes fails with the following
error:

    FailedMount: MountVolume.MountDevice failed for volume "pvc-0e8fdd18-873b-4420-bd27-fa6c02a49496" : rpc error: code = Internal desc = subvolume csi-vol-0d68d71a-1f5f-11eb-96d2-0242ac110012 has unsupported quota: infinite

By ignoring the quota of Infinite, and not setting a quota in the
Subvolume object, this problem should not happen again.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-11-06 14:58:26 +00:00
John Mulligan
8a41cd03a5 journal: fix reading omaps from objects with large key counts
The implementation of getOMapValues assumed that the number of key-value
pairs assigned to the object would be close to the number of keys
being requested. When the number of keys on the object exceeded the
"listExcess" value the function would fail to read additional keys
even if they existed in the omap.
This change sets a large fixed "chunk size" value and keeps reading
key-value pairs as long as the callback gets called and increments
the numKeys counter.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2020-11-06 06:42:22 +00:00
Mudit Agarwal
0ecfd0e72c rbd: replace go-ceph GetParentInfo() with GetParent()
GetParent() is a newer and better version of
GetParentInfo() in go-ceph.

Signed-off-by: Mudit Agarwal <muagarwa@redhat.com>
2020-11-03 08:00:12 +00:00
Niels de Vos
9732cf16a1 cephfs: drop unused Credentials from resizeVolume()
When using go-ceph and the volumeOptions.Connect() call, the credentials
are not needed once the connection is established.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-28 08:02:12 +00:00
Niels de Vos
5baed6190c cephfs: implement resizeVolume() with go-ceph
Reduce the number of calls to the `ceph fs` executable to improve
performance of CephFS volume resizing.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-28 08:02:12 +00:00
Niels de Vos
d431402101 cephfs: make resizeVolume() a method of volumeOptions
This prepares resizeVolume() so that the volumeOptions.conn can be used
for connecting with go-ceph and use the connection cache.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-28 08:02:12 +00:00
Niels de Vos
6a46b8f17f cephfs: implement getSubVolumeInfo() with go-ceph
Fixes: #1551
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-23 10:58:35 +00:00
Niels de Vos
429f7acd8f cephfs: make getSubVolumeInfo() a method of volumeOptions
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-23 10:58:35 +00:00
Humble Chirammal
992d4b92bb util: NewCredentials() dont have any callers
We have below exported function in credentials.go which is not
called from anywhere in the repo. Removing it for the same reason.

```
 // NewCredentials generates new credentials when id and key
 // are provided.

 func NewCredentials(id, key string) (*Credentials, error) {
...
```

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-10-21 21:04:11 +00:00
Humble Chirammal
5ac3f1e29e util: change member cap of CSIDriver struct
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-10-21 12:30:18 +00:00
Humble Chirammal
25617929f0 rbd: use different variable instead of builtin cap function
`cap` builtin function returns the capacity of a type. Its not
good practice to use this builtin function for other variable
names, removing it here

Ref# https://golang.org/pkg/builtin/#cap

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-10-21 12:30:18 +00:00
Humble Chirammal
2d70e25081 util: remove unused CSI server initialization functions
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-10-21 11:17:22 +00:00
Niels de Vos
5a1e370433 util: drop nolint comment for execCommandJSON()
golang-ci suddenly complains about the following issue

    internal/cephfs/util.go:41:1: directive `// nolint:unparam //  todo:program values has to be revisited later` is unused for linter unparam (nolintlint)
    // nolint:unparam //  todo:program values has to be revisited later
    ^

Dropping the comment completely seems to fix it. Ideally
execCommandJSON() will get removed once the migration to go-ceph is
complete.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-20 15:38:44 +00:00
Niels de Vos
eb2584095b cephfs: implement getMetadataPool() with go-ceph
Fixes: #1554
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-20 15:38:44 +00:00
Niels de Vos
cabdac4913 cephfs: make getMetadataPool() a method of volumeOptions
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-20 15:38:44 +00:00
Humble Chirammal
70358c8eb7 rbd: volJournal.Connect() return wrongly pushed to caller
volJournal.Connect() got the error on err2 variable, however
the return was on variable err which hold the error return of
DecomposeCSIID() which is wrong. This cause the error return wrongly
parsed and pushed from the caller. From now on, we are reusing the
err variable to hold and revert the error of volJournal.Connect().

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-10-20 12:45:51 +00:00
Humble Chirammal
5c73f0e41b rbd: correct the code comment for ErrFlattenInProgress
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-10-20 08:59:25 +00:00
Humble Chirammal
9dee064b77 cephfs: fix wrong error check in CreateVolume rollback action
Previously the purgeVolume error was ignored due to wrong error variable
check in the createVolume. With this change it checks on the proper error.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-10-20 04:08:51 +00:00
Humble Chirammal
5b537754a7 cephfs: replace variable name for builtin cap attribute
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-10-19 13:25:15 +00:00
Niels de Vos
0e96199a1c cephfs: free resources when newVolumeOptionsFromVolID() errors out
The allocated, and potentially connected, volumeOptions object in
newVolumeOptionsFromVolID() is not cleaned-up in case of errors. This
could cause resource leaks.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-19 09:15:43 +00:00
Niels de Vos
0f9087d05e cephfs: connect snapshot to Ceph cluster in newSnapshotOptionsFromID()
Without connection, follow-up oparations on the volumeOptions object
will cause a panic. This should fix a regression in CephFS testing.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-19 09:15:43 +00:00
Niels de Vos
157d2aff64 cephfs: implement getFsName() with go-ceph
Fixes: #1549
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-15 13:54:57 +00:00
Niels de Vos
0a3f9d3487 cephfs: make getFsName() a method of volumeOptions
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-15 13:54:57 +00:00
Niels de Vos
f646840779 cephfs: implement getFscID() with go-ceph
Fixes: #1550
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-15 12:33:34 +00:00
Niels de Vos
b28ff63a29 cephfs: make getFscID() a method of volumeOptions
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-15 12:33:34 +00:00
Humble Chirammal
75c6cb6544 rbd: remove unused credentials argument from StoreImageID()
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-10-15 06:46:41 +00:00
Humble Chirammal
4cb7d2bbbd rbd: remove unwanted condition check in flattenCloneImage
All the previous condition checks exit from the function and
when it reach to this block its obvious that error is non nil,
we dont need an extra check here.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-10-15 06:20:47 +00:00
Niels de Vos
5598c279da cleanup: no magic number for RWX permissions
golang-ci complains that 0777 is a magic number, so make it a constant
with a clear description.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-14 15:16:56 +00:00
Niels de Vos
9bd10d2698 cephfs: newVolumeOptions() does not need all secrets, only credentials
There is no need to pass all secrets on to newVolumeOptions(), it only
needs the credentials. As the caller of newVolumeOptions() already has
the credentials generated, just pass them along instead of the raw
secrets.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-14 15:16:56 +00:00
Niels de Vos
66046e4312 cephfs: drop credentials from createVolume()
The credentials are not used anymore, the volume object is already
connected to the cluster when createVolume() is called.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-14 15:16:56 +00:00
Niels de Vos
8cba08189d cephfs: implement createVolume() with go-ceph
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-14 15:16:56 +00:00
Niels de Vos
fe62348cfb util: add ClusterConnection.GetFSAdmin()
The go-ceph CephFS Admin API needs a rados.Conn to connect the Admin
interface to the Ceph cluster. As the rados.Conn is internal to the
ClusterConnection type, add GetFSAdmin() to create a new FSAdmin
instance for a specific connection.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-14 15:16:56 +00:00
Niels de Vos
0f108edc7b cephfs: connect to the Ceph Cluster with go-ceph
Add the ClusterConnection to the volumeOptions type, so that future use
of go-ceph can connect to the Ceph cluster.

Once a volumeOptions object is not needed anymore, it needs to get
destroyed to free associated resources like the ClusterConnection.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-14 15:16:56 +00:00
Humble Chirammal
be9195cf32 rbd: remove false error condition check in checkSnapCloneExists
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-10-13 12:56:51 +00:00
Humble Chirammal
7992033156 rbd: remove false error condition check in rbdVol.Exists()
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-10-13 12:56:51 +00:00
Humble Chirammal
c9b7b6561a rbd: remove false error condition check in genVolFromVolID
The error check condition in genVolFromID() is always false as far as
code reading/workflow goes. Removing it for the same reason.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-10-13 12:56:51 +00:00
Humble Chirammal
364b3afcaa rbd: correct typo in validateRequestedVolumeSize()
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-10-13 08:58:05 +00:00
Humble Chirammal
36ee8b2240 rbd: make sure csi rbd pv encryption defaulting to luks2 version
Fixes: #https://github.com/ceph/ceph-csi/issues/1564

As per release notes this new version is compatible with previous
version of LUKS.
https://www.saout.de/pipermail/dm-crypt/2017-December/005771.html

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-10-13 05:11:25 +00:00
Niels de Vos
f6de345cc9 cleanup: standardize error logs in cephfs/volume.go
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-10-01 12:00:59 +00:00
Humble Chirammal
b5d1edd3f7 cephfs: correct cephFSCloneInprogress source code comment
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-09-25 17:36:14 +00:00
Marton Natko
91c49c14b3 rbd: fix problem with deletion of a namespaced image
fix problem with deletion of a namespaced image

Signed-off-by: Marton Natko <mnatko@transip.nl>
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-09-25 15:35:22 +00:00
Marton Natko
2fbca0d1e2 rbd: fix typo in createVolume validation
Signed-off-by: Marton Natko <mnatko@transip.nl>
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-09-25 15:35:22 +00:00
Mudit Agarwal
e1237f348f rbd: replace cli with go-ceph for snapshot namespace
cephcsi uses cli for fetching snap list as well as to check the
snapshot namespace, replaced that with go-ceph calls.

Signed-off-by: Mudit Agarwal <muagarwa@redhat.com>
2020-09-25 08:07:32 +00:00
Madhu Rajanna
89b326c896 cephfs: check ENOTEMPTY when removing subvolume
return a proper error message to the user when
the subvolume has the snapshots and it cannot
be removed until the snapshots on the subvolume
have to be deleted.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-09-23 13:51:23 +00:00
Niels de Vos
6f2f972f5b cephfs: trim spaces around fuseMountOptions
When passing

    fuseMountOptions: debug

in the StorageClass, the mount options passed on the ceph-fuse
commandline result in "-o nonempty ,debug". The additional space before
the ",debug" causes the mount command to fail.

Fixes: 1485
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-09-23 12:12:51 +00:00
Madhu Rajanna
d1f175d9f3 rbd: add support for rbd map and unmap options
added support for providing map and unmap
options to rbd CLI when mapping rbd image
on the node.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-09-21 13:27:28 +00:00
Yug
0682f0e500 cephfs: Fetch subvolume features in newSnapshotOptionsFromID
Get subvolume features using subvolume info and update the
volume options with Features parameter.

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-09-16 16:05:28 +00:00
Yug
9e959c45a3 cephfs: No-Op Unprotect call if "snapshot-autoprotect" feature present
The subvolume features consists the list of features, which if includes
"snapshot-autoprotect", result in query based approach to
detecting the need for protect/unprotect.
If "snapshot-autoprotect" feature is present, The UnprotectSnapshot
call should be treated as a no-op

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-09-16 16:05:28 +00:00
Yug
43a4c100ec cephfs: No-Op Protect call if "snapshot-autoprotect" feature present
The subvolume features consists the list of features, which if includes
"snapshot-autoprotect", result in query based approach to
detecting the need for protect/unprotect.
If "snapshot-autoprotect" feature is present, The ProtectSnapshot
call should be treated as a no-op

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-09-16 16:05:28 +00:00
Yug
365eb58ff7 cephfs: use subvolume info if available
Use subvolume info to fetch the subvolume path.
If `subvolume info` command is not available,
use `getpath` command instead.

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-09-10 15:03:45 +00:00
Yug
59fc4aa00f cephfs: validate if subvolume has snapshot-retention feature
Add validateSnapshotRetention() to verify if subvolume has
`snapshot-retention` feature.

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-09-10 15:03:45 +00:00
Yug
6351358752 cephfs: Add --retain-snapshots flag to purge volume operations
Snapshots can be retained even after subvolume deletion in
Ceph 14.2.12. Adding support for the same in ceph-csi.

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-09-10 15:03:45 +00:00
Madhu Rajanna
860b65c446 cephfs: allow readonly only for volume with contentsource
It doesnot make sense to allow the creation of empty
volumes with readonly access, this commit allows the
creation of volume which is having readonly capabilities
only if the content source is set for the volume.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-09-10 13:47:23 +00:00
Madhu Rajanna
2deabcd284 rbd: allow readonly only for volume with contentsource
It doesnot make sense to allow the creation of empty
volumes which is going to be accessed with readonly mode,
this commit allows the creation of volume which is having
readonly capabilities only if the content source is set
for the volume.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-09-10 13:47:23 +00:00
Madhu Rajanna
d816df4f42 util: add helper to validate ROX request
added a helper function to validate is create
request for ROX is for volume from source.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-09-10 13:47:23 +00:00
Humble Chirammal
edca318828 rebase: update the CSI Spec to v1.3.0
This spec add the extra capability to  node and controller
volume to report volume condition of a pv..etc.

Refer # https://github.com/ceph/ceph-csi/issues/1356

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-09-04 09:24:01 +00:00
Mudit Agarwal
4de1abad5e rbd: NodeExpandVolume() should use StagingTargetPath
Form kubernetes v1.19 onwards NodeRequest is getting volume path
in StagingTargetPath instead of VolumePath, cephcsi should also
use the same.

Signed-off-by: Mudit Agarwal <muagarwa@redhat.com>
2020-08-25 15:58:44 +00:00
Mudit Agarwal
8c2bdbc001 rbd: implement rbd.updateVolWithImageInfo() with go-ceph
updateVolWithImageInfo() is currently executing an rbd command
to get details about an RBD image. Replaced it with the
required go-ceph functions.

Signed-off-by: Mudit Agarwal <muagarwa@redhat.com>
2020-08-25 07:06:40 +00:00
Madhu Rajanna
38892c3068 rbd: rearrange the cleanup order incase clone fails
in case of clone failure, we need to first delete
the clone and the snapshot from which we created
the clone, then as part of cleanup we need to remove
the temporary cloned image and the temporary snapshot
created on the parent image.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-24 08:16:59 +00:00
Madhu Rajanna
a7b65087af rbd: fix return error message
incase of flattening error return the
flatten error.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-24 08:16:59 +00:00
Madhu Rajanna
0bd45a5eb1 cephfs: use string const for cli error comparison
we should not return the CLI errors in GRPC errors
we need to return proper readable error messages
to the user for better understanding and better
debugging.

updates #1242

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-19 17:09:12 +00:00
Madhu Rajanna
27d2cc644d util: replace klog with util logger in liveness.go
replace klog with util logger in liveness.go

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-19 14:38:12 +00:00
Madhu Rajanna
ed289fea5e journal: replace klog with util logger in voljournal.go
journal: replace klog with util logger in voljournal.go

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-19 14:38:12 +00:00
Madhu Rajanna
1a9cd23f64 journal: replace klog with util logger in omap.go
replace klog with util logger in omap.go

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-19 14:38:12 +00:00
Madhu Rajanna
7c9fee17af rbd: replace klog with util logger in snapshot.go
rbd: replace klog with util logger in snapshot.go

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-19 14:38:12 +00:00
Madhu Rajanna
9305e27797 rbd: replace klog with util logger in rbd_util.go
rbd: replace klog with util logger in rbd_util.go

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-19 14:38:12 +00:00
Madhu Rajanna
90cc5a4fd0 rbd: replace klog with util logger in rbd_journal.go
replace klog with util logger in rbd_journal.go

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-19 14:38:12 +00:00
Madhu Rajanna
9512aa6206 rbd: replace klog with util logger in rbd_attach.go
replace klog with util logger in rbd_attach.go

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-19 14:38:12 +00:00
Madhu Rajanna
e768c0dfc0 rbd: replace klog with util logger in nodeserver.go
replace klog with util logger in nodeserver.go

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-19 14:38:12 +00:00
Madhu Rajanna
883485b082 rbd: replace klog with util logger in driver.go
replace klog with util logger in driver.go

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-19 14:38:12 +00:00
Madhu Rajanna
c4f30c23c1 rbd: replace klog with util logger in clone.go
replace klog with util logger in clone.go

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-19 14:38:12 +00:00
Madhu Rajanna
d8ca56fbe9 rbd: replace klog with util in controllerserver.go
replace klog with util in controllerserver.go

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-19 14:38:12 +00:00
Humble Chirammal
1f5b84745f rbd: Bail out from nodeexpansion if its block mode pvc
At CSI spec < 1.2.0, there was no volumecapability in the
expand request. However its available from v1.2+ which allows
us to declare the node operations based on the volume mode.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-19 12:34:20 +00:00
Madhu Rajanna
787ba69089 cephfs: fix lock release in volumeDelete
currently the lock is not released which is
taken on the request name. this is causing issues
when the subvolume is requested for delete.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-18 06:12:43 +00:00
Madhu Rajanna
31c7c95fdb cephfs: check error output contains Error: ENOENT
execCommandErr returns both error and stderror
message. checking strings.HasPrefix is not helpful
as the stderr will be the first string. its good
to do string comparison and find out that error
is volume not found error.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-18 06:12:43 +00:00
Madhu Rajanna
f566b0fc2d cephfs: replace Warningf with WarningLog in unmountVolume
replace Errorf with ErrorLog in unmountVolume
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
3fe3d25dda cephfs: replace Errorf with ErrorLogMsg in loadAvailableMounters
replace Errorf with ErrorLog in loadAvailableMounters
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
fbe090065d cephfs: replace Errorf with ErrorLog in purgeVolume
replace Errorf with ErrorLog in purgeVolume
for context based logging

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
5a1652e5e8 cephfs: replace Errorf with ErrorLog in resizeVolume
replace Errorf with ErrorLog in resizeVolume
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
a882c1194b cephfs: replace Errorf with ErrorLog in createVolume
replace Errorf with ErrorLog in createVolume
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
f3d9b8933b cephfs: replace Errorf with ErrorLog in getSubVolumeInfo
replace Errorf with ErrorLog in getSubVolumeInfo
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
2b7aade289 cephfs: replace Errorf with ErrorLog in getVolumeRootPathCeph
replace Errorf with ErrorLog in getVolumeRootPathCeph
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
d1ac832266 cephfs: replace Errorf with ErrorLog in genSnapFromOptions
replace Errorf with ErrorLog in genSnapFromOptions
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
e83a51c1e1 cephfs: replace Errorf with ErrorLog in parseTime
replace Errorf with ErrorLog in parseTime
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
e7a043271d cephfs: replace Errorf with ErrorLog in cloneSnapshot
replace Errorf with ErrorLog in cloneSnapshot
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
841a3713f5 cephfs: replace Errorf with ErrorLog in unprotectSnapshot
replace Errorf with ErrorLog in unprotectSnapshot
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
91811c18c5 cephfs: replace Errorf with ErrorLog in protectSnapshot
replace Errorf with ErrorLog in protectSnapshot
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
72dfe6a265 cephfs: replace Errorf with ErrorLog in getSnapshotInfo
replace Errorf with ErrorLog in getSnapshotInfo
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
7e8cc41aca cephfs: replace Errorf with ErrorLog in deleteSnapshot
replace Errorf with ErrorLog in deleteSnapshot
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
7f00fa66f4 cephfs: replace Errorf with ErrorLog in createSnapshot
replace Errorf with ErrorLog in createSnapshot
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
57454cf379 cephfs: replace Errorf with ErrorLog in nodeUnstageVolume
replace Errorf with ErrorLog in nodeUnstageVolume
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
0a5581b7a0 cephfs: replace Errorf with ErrorLog in NodeUnPublishVolume
cephfs: replace Errorf with ErrorLog in NodeUnPublishVolume
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
79dbc41d14 cephfs: replace Errorf with ErrorLog in NodePublishVolume
replace Errorf with ErrorLog in NodePublishVolume
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
48a313c6cc cephfs: replace Errorf with ErrorLog in mount
replace Errorf with ErrorLog in mount for
context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
032901b8d0 cephfs: replace Errorf with ErrorLog in nodeStageVolume
replace Errorf with ErrorLog in nodeStageVolume
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
c57189382f cephfs: replace Errorf with ErrorLog in checkSnapExists
replace Errorf with ErrorLog in checkSnapExists
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
98e97baa60 cephfs: replace Errorf with ErrorLog in checkVolExists
replace Errorf with ErrorLog in checkVolExists
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
abd548e420 cephfs: replace warning with WarningLogMsg in Run
replace warning with WarningLogMsg in Run for
context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
38273f879f cephfs: replace fatal with util.FatalLog in Run
replace fatal with util.FatalLog in Run for
context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
eeb0859f99 util: rename FatalLog to FatalLogMsg for parity
rename FatalLog to FatalLogMsg to keep functions
in parity functions ends with Msg if its only message
logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
71a6dc3ba8 cephfs: replace Errorf with util.ErrorLog in getCloneInfo
replace Errorf with util.ErrorLog in getCloneInfo
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
ae6b77f105 cephfs: add comment for exported CloneStatus struct
added comment for exported structure.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
18d65ba3f5 cephfs: replace Errorf with ErrorLog in createCloneFromSnapshot
replace Errorf with ErrorLog in createCloneFromSnapshot
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
3fa512617b cephfs: replace Errorf with util.ErrorLog in cleanupClone
replace Errorf with util.ErrorLog in cleanupCloneFromSubvolumeSnapshot
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
e1fb708d48 cephfs: replace Errorf with util.ErrorLog in createCloneFromSubvolume
replace Errorf with util.ErrorLog in createCloneFromSubvolume
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
20dbbe54b0 cephfs: replace klog.Warningf with util.WarningLog in deleteSnapshot
replace klog.Warningf with util.WarningLog in deleteSnapshot
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
7eefadf1eb cephfs: replace klog.Warning with util.WarningLog in DeleteVolume
replace klog.Warning with util.WarningLog in DeleteVolume
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
e3c3bbf0e3 cephfs: replace klog.Warning with util.WarningLog in CreateVolume
replace klog.Warning with util.WarningLog in CreateVolume
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
3d12456493 util: add WarningLog to log with context
added WarningLog to log with context

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
b223924044 util: rename WarningLog to WarningLogMsg
renamed WarningLog function to WarningLogMsg
to log with message.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
f45ad230a2 cephfs: replace klog.Error with util.ErrorLog in deleteSnapshot
replace klog.Error with util.ErrorLog in deleteSnapshot
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
52f14cf9fe cephfs: replace klog.Error with util.ErrorLog in validateSnapshotReq
replace klog.Error with util.ErrorLog in validateSnapshotReq
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
d101cbc888 cephfs: replace klog.Error with util.ErrorLog in doSnapshot
replace klog.Error with util.ErrorLog in doSnapshot
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
5802265b28 cephfs: replace klog.Error with util.ErrorLog in createSnapshot
replace klog.Error with util.ErrorLog in createSnapshot
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
81b52b311d cephfs: replace klog.Error with util.ErrorLog in ControllerExpand
replace klog.Error with util.ErrorLog in ControllerExpand
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
7ed5dfe81a cephfs: replace klog.Error with util.ErrorLog in DeleteVolume
replaced klog.Error with util.ErrorLog in DeleteVolume
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
cc600dab76 cephfs: replace klog.Error with util.ErrorLog in CreateVolume
replaced klog.Error with util.ErrorLog
in createVolume function.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
3c45ec32db cephfs: replace klog.Error in createBackingVolume
replace klog.Error with util.ErrorLog in
createBackingVolume function.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
8c32045c4e util: use ErrorLog for context based logging
use ErrorLog for context based logging for
more detailed logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
7c24f4d597 util: add ErrorLog for log with context
added a new function ErrorLog to log with
context.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
9229e84a77 util: rename ErrorLog to ErrorLogMsg to parity
as we have 2 functions for logging. one for logging
with message and another one is for logging with
context. renamed ErrorLog to  ErrorLogMsg to log
with messages.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
d4186a6880 rbd: discard flattening error if the image doesnt have parent
if the image is created without flattening image-feature
the image will get few image-features by default, deep-flatten
is one of them. if the image doesnot have any parent
the rbd image flattening will fail, This commit discards
error message if the image doesnt have any parent.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 08:37:53 +00:00
Madhu Rajanna
8a581ec6c0 rbd: fix build issue for rbd
merging of https://github.com/ceph/ceph-csi/pull/1035
broken the cephcsi building. This commits fixes
the build issue.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-12 19:53:48 +05:30
Mehdy Khoshnoody
3081eabba5 doc: Add rados namespace to option to csi-configs
A minimal documentation on how to use a rados namespace with rbd.

Signed-off-by: Mehdy Khoshnoody <mehdy.khoshnoody@gmail.com>
2020-08-12 16:22:58 +05:30
Mehdy Khoshnoody
fc5eadf106 rbd: Add rados namespace support for rbd
Make sure to operate within the namespace if any given
when dealing with rbd images and snapshots and their journals.

Signed-off-by: Mehdy Khoshnoody <mehdy.khoshnoody@gmail.com>
2020-08-12 16:22:58 +05:30
Madhu Rajanna
bfde065f92 util: move getMonsAndClusterID to util
as we had duplicate functions in both cephfs
and rbd this commit moves the function to util.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-11 10:02:53 +00:00
Madhu Rajanna
18f4e9d519 util: move csiconfigfile to util
as we have csiconfigfile in both cephfs
and rbd moving the configfile path to
util folder.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-11 10:02:53 +00:00
Niels de Vos
b864afd6d7 util: use local ErrorLog() for Create/RemoveObject() instead of klog
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-08-11 08:11:37 +00:00
Niels de Vos
eaea18df0f util: use local FatalLog() for NewK8sClient() instead of klog
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-08-11 08:11:37 +00:00
Niels de Vos
9d4020d255 util: use local ErrorLog() for CheckKernelSupport() instead of klog
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-08-11 08:11:37 +00:00
Niels de Vos
75ae16cfab util: use local DefaultLog() for util/topology instead of klog
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-08-11 08:11:37 +00:00
Niels de Vos
bf02227207 util: use local ErrorLog() for OperationLock.release() instead of klog
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-08-11 08:11:37 +00:00
Niels de Vos
fb56a7595a util: use local ErrorLog() for GetCryptoPassphrase() instead of klog
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-08-11 08:11:37 +00:00
Niels de Vos
61924adf57 util: use FatalLog() in util/httpserver.go
Hide the use of klog by using our own FatalLog() function.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-08-11 08:11:37 +00:00
Niels de Vos
ab033f85df util: add FatalLog(), ErrorLog() and WarningLog() functions
These functions will always log the message, irrespectively of the
log-level that has been configured.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-08-11 08:11:37 +00:00
Niels de Vos
44863a9d29 cleanup: move log functions to util/log.go
There is a util/log.go file, so the log functions in util/util.go can be
moved there.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-08-11 08:11:37 +00:00
Madhu Rajanna
b86d329d1e rbd: add stagingpath to error message
if the stagingpath is not present inside
the container send the staging path in error
message.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-10 08:51:23 +00:00
Humble Chirammal
7d2bba1375 cephfs: introduce parsetime() to parse createdAt field in snap return
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-08 00:11:35 +00:00
Humble Chirammal
c3400bfb97 cephfs: add snapshot create and delete functionalilies
It also add helper routines like parsetime,doSnapshot..etc

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-08 00:11:35 +00:00
Humble Chirammal
20c90ddfc4 cephfs: getVolumeRootPathCeph and purgeVolume use new error strings
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-08 00:11:35 +00:00
Humble Chirammal
1f4839cb5e cephfs: add subvolume struct and getSubvolumeInfo calls
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-08 00:11:35 +00:00
Humble Chirammal
31e9288f8f cephfs: remove inValidCommmand and errNotFoundString
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-08 00:11:35 +00:00
Humble Chirammal
5bceb590fd cephfs: add snap reserve/unreserve and snap exist functionalities
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-08 00:11:35 +00:00
Humble Chirammal
38d005e4e6 cephfs: validate create volume request
Also add functionality to generate snap from request
and to get mon and clusterID from the request

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-08 00:11:35 +00:00
Humble Chirammal
9c000add29 cephfs: Change checkVolExist for snapshot and clone workflow
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-08 00:11:35 +00:00
Humble Chirammal
c773097f85 cephfs: add snapshot and clone helper functions
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-08 00:11:35 +00:00
Humble Chirammal
d1fe12b4f0 cephfs: change createvolume for snapshot/clone workflow
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-08 00:11:35 +00:00
Humble Chirammal
c31f349c6f cephfs: use delete operation lock in delete volume
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-08 00:11:35 +00:00
Humble Chirammal
f9257ed1cb cephfs: add snapshot create/delete capabilities in controllerserver
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-08 00:11:35 +00:00
Humble Chirammal
5a71949dcd cephfs: introduce newSnapshotOptionsFromID to generate volOpt and sid
From provided CSI volume ID this populate volumeOptions and snapshot
identifier after connecting to the snapJournal.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-08 00:11:35 +00:00
Humble Chirammal
a4f2c5617c cephfs: retrieve NewCSISnapshotJournal with namespace set
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-08 00:11:35 +00:00
Humble Chirammal
1746b4e6ee cephfs: add various error strings for command validation
Compared to previous version of the error strings, this change
depend on error strings like ENOENT, EEXIST, EINVAL..etc
The format of the error strings change in different cluster versions
and the error code return should not change. This also add extra
error strings for snapshot and clone operation outputs

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-08 00:11:35 +00:00
Humble Chirammal
6dcf03a4bc cephfs: add checkContentSource() to validate the data source
checkContentSource() validate the data source in the request
and then populate volumeOptions or snapshotshot identifier in
case of snapshot source. If the data source is volume, then
parentVolumeOption and pvID are populated.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-08 00:11:35 +00:00
Humble Chirammal
2bb0bbde7f cephfs: change createBackingVolume to accomodate clone operations
adjust createBackingVolume() to create a subvolume from snapshot
or existing subvolume by taking restore or clone operation locks

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-08 00:11:35 +00:00
Humble Chirammal
c968380134 cephfs: add snapshotlock and operationlock to controllerserver
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-08 00:11:35 +00:00
Humble Chirammal
66827facc3 cephfs: make use of expand operation lock in ControllerExpandVolume
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-08 00:11:35 +00:00
Madhu Rajanna
2458ec6573 rbd: return error if fetching cluster id fails
if we are not able to fetch the cluster-ID from
the createSnapshot request and also if we are
not able to get the monitor information from
the cluster-ID return error instead of using
the parent image information.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-03 14:25:06 +00:00
Mudit Agarwal
9ed0811422 rbd: implement rbdVolume.resize() with go-ceph
Replaced command execution with go-ceph Resize() function.
Volsize is being updated before waiting for resize() to return,
fixed it to get updated only after resize() is successful.

Signed-off-by: Mudit Agarwal <muagarwa@redhat.com>
2020-08-03 10:50:01 +00:00
Niels de Vos
be9e7cf956 rbd: pass context.Context to rbdVolume.resize()
While adding the context.Context to the resizeRBDimage() function, it
became a little ugly. So renaming the function to resize() and making it
a method of the rbdVolume type.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-24 16:04:13 +00:00
Niels de Vos
36469b87e2 util: make ExecComand return stdout and stderr as string
Most consumers of util.ExecCommand() need to convert the returned []byte
format of stdout and/or stderr to string. By having util.ExecCommand()
return strings instead, the code gets a little simpler.

A few commands return JSON that needs to be parsed. These commands will
be replaced by go-ceph implementations later on. For now, convert the
strings back to []byte when needed.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-24 16:04:13 +00:00
Niels de Vos
ddac66d76b util: use context.Context for logging in ExecCommand
All calls to util.ExecCommand() now pass the context.Context. In some
cases this is not possible or needed, and util.ExecCommand() will not
log the command.

This should make debugging easier when command executions fail.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-07-24 16:04:13 +00:00