Commit Graph

1125 Commits

Author SHA1 Message Date
Niels de Vos
290beb4dda cephfs: add kernel version detection for mounting with client
Linux kernel 4.17.0 adds support for quota with CephFS. Without quota,
it is not possible to fullfill the requirements of the CSI Spec and
guarantee sufficient space on the filesystem for a volume. With this in
mind, usage of the kernel client is only allowed with kernel 4.17.0 or
newer.

However, some Linux vendors backport features and patches to their
Enterprise products. These kernels may have an older version, but do
support quota. One of these is the kernel that comes with RHEL-7.7.

By comparing the current running version of the Linux kernel against
known versions that support quota, we can now automatically decide to
use the kernel client, or not.

Note that this does not change the 'forcekernelclient' parameter. The
parameter is still available and can be used for kernels that are not in
the 'known to support quota list'. Or users can pass the parameter to
use a CephFS kernel client that does not support quota.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-11-13 11:56:09 +00:00
Madhu Rajanna
dfc3562e29 Add Version flag to cephcsi
This will be helpful if someone wants
to check the cephcsi version

output
```
docker run quay.io/cephcsi/cephcsi:v1.2.1 --version
Cephcsi Version: v1.2.1
Git Commit: 4b871366327d63e27fc1abfb699f0faaf0fc16b9
GoVersion: go1.12.5
Compiler: gc
Platform: linux/amd64
```

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-11-06 12:35:28 +00:00
techhanx
8e117db80d Change e2e rook-ceph default version to 'v1.1.2' not master
When rook-ceph is upgraded and changed some feature, e2e can be
failed. Change rook-ceph default verion to 'v1.1.2' explicitly
which is working fine in current code.
2019-10-21 08:40:58 +00:00
techhanx
e4b4c70d92 Delete IDE releated files in .gitignore
The IDE is a user preference, and they should configure in
individual envrionment. Delete IDE releated files in the
gitignore.
2019-10-17 08:17:27 +00:00
Stefan Haas
6a2717ce20 Added forcecephkernelclient as startup parameter to force enabling ceph
Signed-off-by: Stefan Haas <shaas@suse.com>
2019-10-16 06:47:10 +00:00
Madhu Rajanna
50268d5947 Fix issue in helm chat pushing
From master and release-v1.2.0 branch we will be
pushing the canary charts. when we are doing the release
i.e v1.2.2 from release-v1.2.0 branch we will replace
all canary to released tag v1.2.2 and push helm charts
for v.1.2.2 and will revert back the changes in
release-v1.2.0 branch push canary tagged charts(
this will be same logic as pushing the container image)

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-10-15 11:10:19 +05:30
Madhu Rajanna
52c4d29f65 Remove csi-v0.3 from deploy.sh
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-10-15 11:10:19 +05:30
Jason Dillaman
a274b19bfa Handle EACCESS error from 'ceph rbd task add remove'
If the RBD user does not have permissions to talk to the
Ceph MGR, it should gracefully fallback to the slower foreground
image deletion.

Fixes: #677
Signed-off-by: Jason Dillaman <dillaman@redhat.com>
2019-10-13 14:50:40 +00:00
Madhu Rajanna
239822f147 reuse existing code for size Roundoff
This PR addresses the review comments in
https://github.com/ceph/ceph-csi/pull/644

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-10-11 11:07:39 +00:00
Madhu Rajanna
7274bd09e5 Fix volsize for cephfs and rbd
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-10-11 08:22:27 +00:00
Madhu Rajanna
9287948991 update registration directory name
updated cephfs registration directory
name to match with rbd implementaion

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-10-10 07:16:09 +00:00
Humble Chirammal
1efdf14ac5 At present, the request timeout of sidecars are at the 60s and this is a request to increase
this time out value to 150s or higher. The higher timeout value can help to reduce the
load of our backend ceph cluster and also can avoid throttling issues at sidecars to an extent.

Fix# #602

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2019-10-09 05:28:40 +00:00
Ilya Dryomov
9a61fa0c0a Format ext4 with lazy_journal_init
Skip zeroing of the journal on freshly created images.  As only dynamic
PVs are supported, it is fine to assume that existingFormat == "" image
has never been mapped and written to before.

lazy_itable_init is enabled by default, passing it for consistency with
lazy_journal_init.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2019-10-04 15:07:48 +00:00
Daniel-Pivonka
cd52798a51 Change default csi liveness ports to ones less common
Signed-off-by: Daniel-Pivonka <dpivonka@redhat.com>
2019-10-01 15:08:58 +00:00
Madhu Rajanna
3784268dde update mergify for auto merging
update rules to auto merge in master branch

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-09-30 17:19:00 +05:30
wilmardo
abdadef8bc fix: add POD_NAMESPACE to RBD provisioner deployments
Signed-off-by: wilmardo <info@wilmardenouden.nl>
2019-09-30 11:15:35 +00:00
Madhu Rajanna
0b261e2819 if there are changes requested dont merge PR
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-09-27 16:11:33 +05:30
Madhu Rajanna
ed364cf6ea Replace ' with " in mergify.yaml
as per the examples rules in
https://doc.mergify.io/examples.html, string are
surrounded  by `""`, it better to follow same
format here also.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-09-27 16:11:33 +05:30
Madhu Rajanna
136d81b736 Add mergify rule to auto merge backported PR
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-09-27 16:11:33 +05:30
Madhu Rajanna
3d0cba1931 Remove rootfs from rbd provisioner pod
rootfs dependency was removed from rbd
by removing support for `nsenter`, This
PR removed the `/` mount from provisioner

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-09-27 06:29:56 +00:00
wilmardo
6ee381db3a refactor: Merge 1.13 and 1.14 Helm charts and improve charts
Signed-off-by: wilmardo <info@wilmardenouden.nl>
2019-09-27 05:49:18 +00:00
Madhu Rajanna
b8568a5bb9 Add a check for nil secrets
Improve the error message if
secrets are not provided in request

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-09-27 05:10:01 +00:00
Madhu Rajanna
ceef9ff3a5 Backport PR when label is set
backport PR to release-v1.2.0 branch
from master branch  when label is set
on PR

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-09-27 09:50:59 +05:30
Humble Devassy Chirammal
279197bc78
Merge pull request #635 from humblec/ext4
Set default FS to "ext4" on RBD device
2019-09-25 14:06:13 +05:30
Humble Chirammal
b44a81bdbc We were defaulting to ext4 at first and then moved toxfs.
However further testing shows that, ext4 should be
the default or preferred fs for RBD devices.

This patch bring that change

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2019-09-24 14:36:50 +05:30
Madhu Rajanna
d4a67c05f3 Remove nsenter packages from vendor
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-09-23 13:22:29 +00:00
Madhu Rajanna
70d49b4e47 tempate changes for containerized flag removal
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-09-23 13:22:29 +00:00
Madhu Rajanna
7002da1a6a deprecate containerized flag and remove nsenter
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-09-23 13:22:29 +00:00
Madhu Rajanna
69b7c4672f Added E2E for rbd nodeplugin restart
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-09-23 13:22:29 +00:00
Humble Chirammal
0e6617e1ff Format RBD volumes with nodiscard formatting options.
Currently rbd CSI plugin uses formatAndMount of
mount.SafeFormatAndMount. This does not allow to pass or use
specific formatting arguments with it. This patch introduce
RBD specific formatting options with both xfs and ext4,
for example: -E no-discard with ext4 and -k option with
XFS to boost formatting performance of RBD device.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2019-09-20 13:16:53 +00:00
Madhu Rajanna
e2890a27ff connect to provisioner socket
Fixes: #619

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-09-20 08:13:19 +00:00
Madhu Rajanna
6aac399075 Change the logic of locking
if any on going opearation is seen,we
have to return Abort error message

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-09-20 07:37:17 +00:00
Madhu Rajanna
18151e9ca8 use rook v1.1.0 in E2E testing
rook master is not good enough to
depend on for E2E testing, swithing to v1.1.0
for E2E

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-09-18 12:30:43 +00:00
Madhu Rajanna
6da96c6327 remove support for create image with image-format 1
tried to create an image with image-format=1

```
sh-4.2# rbd create --size=1024 replicapool/test --image-format=1
rbd: image format 1 is deprecated
rbd: create error: (22) Invalid argument
2019-09-11 07:00:54.531 7fb0e40bfb00 -1 librbd: Format 1 image creation unsupported.
```

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-09-12 07:52:32 +00:00
Madhu Rajanna
e395080cdc Add req-ID to logging
with this log format we can easily identify
the logs per request

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-09-11 13:45:40 +00:00
Madhu Rajanna
ed9330d2f6 rename Key to CtxKey
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-09-11 13:45:40 +00:00
Madhu Rajanna
41b701c98c Add support for erasure pool in rbd
Allow specifying different metadata and data pools in a
CSI RBD StorageClass

Fixes: #199
Fixes: https://github.com/rook/rook/issues/2650
Fixes: https://github.com/rook/rook/issues/3763

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-09-11 06:48:08 +00:00
Madhu Rajanna
8992e9d321 update readme for v1.2.0 release
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-09-10 13:11:53 +00:00
KingJ
0639e00705 Reorder kernel version checking logic 2019-09-07 11:10:27 +00:00
KingJ
197c8fcfcc Consider Kernel >=5.x as sufficent for using the Kernel mounter 2019-09-07 11:10:27 +00:00
Poornima G
060ff8d25e Add mount option for Cephfs
The storage class already takes MountOptions(MountFlags), these are the
bind mount options. Some of these options may not be recognised by the
cephfs mount. Hence added a new parameterin Storage Class for
- cephfs kernel mount options,
- ceph-fuse mount options

Ceph kernel mount options are different from ceph-fuse options, hence
added two different parameters.

Signed-off-by: Poornima G <pgurusid@redhat.com>
2019-09-06 16:32:10 +00:00
Daniel-Pivonka
d1952e5fd0 Fix liveness connection opening endless sockets
Signed-off-by: Daniel-Pivonka <dpivonka@redhat.com>
2019-09-06 03:15:28 +00:00
Madhu Rajanna
f4b38228ae Remove volumemounter flag from cephfs
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-09-05 07:20:50 +00:00
Poornima G
90c4d6a451 Cephfs: Use ceph kernel client if kernel version >= 4.17
Ceph kernel client is more performant than ceph fuse client.
The kernel client has Quota support only in the kernel version >=4.17.
Hence use ceph kernel client when the kernel version is >=4.17.

Signed-off-by: Poornima G <pgurusid@redhat.com>
2019-09-05 04:55:05 +00:00
Niels de Vos
d03a352abf e2e: correct log format in execCommandInPod()
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2019-09-04 12:26:55 +00:00
Niels de Vos
8f133e03b8 Add 'gosec' to the static-checks
Run static security scanning tool 'gosec' while testing.

URL: https://github.com/securego/gosec
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2019-09-04 11:48:37 +00:00
Niels de Vos
dd668e59f1 Address security concerns reported by 'gosec'
gosec reports several issues, none of them looks very critical. With
this change the following concerns have been addressed:

[pkg/cephfs/nodeserver.go:229] - G302: Expect file permissions to be 0600 or less (Confidence: HIGH, Severity: MEDIUM)
  > os.Chmod(targetPath, 0777)

[pkg/cephfs/util.go:39] - G204: Subprocess launched with variable (Confidence: HIGH, Severity: MEDIUM)
  > exec.Command(program, args...)

[pkg/rbd/nodeserver.go:156] - G302: Expect file permissions to be 0600 or less (Confidence: HIGH, Severity: MEDIUM)
  > os.Chmod(stagingTargetPath, 0777)

[pkg/rbd/nodeserver.go:205] - G302: Expect file permissions to be 0600 or less (Confidence: HIGH, Severity: MEDIUM)
  > os.OpenFile(mountPath, os.O_CREATE|os.O_RDWR, 0750)

[pkg/rbd/rbd_util.go:797] - G304: Potential file inclusion via variable (Confidence: HIGH, Severity: MEDIUM)
  > ioutil.ReadFile(fPath)

[pkg/util/cephcmds.go:35] - G204: Subprocess launched with variable (Confidence: HIGH, Severity: MEDIUM)
  > exec.Command(program, args...)

[pkg/util/credentials.go:47] - G104: Errors unhandled. (Confidence: HIGH, Severity: LOW)
  > os.Remove(tmpfile.Name())

[pkg/util/credentials.go:92] - G104: Errors unhandled. (Confidence: HIGH, Severity: LOW)
  > os.Remove(cr.KeyFile)

[pkg/util/pidlimit.go:74] - G304: Potential file inclusion via variable (Confidence: HIGH, Severity: MEDIUM)
  > os.Open(pidsMax)

URL: https://github.com/securego/gosec
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2019-09-04 11:48:37 +00:00
Madhu Rajanna
d6f1c938d8 comment yum update from dockerfile
currently we are facing issue in  building
docker image,commenting yum update it fix it

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-09-04 11:12:07 +00:00
Madhu Rajanna
64ca401a51 Fix mon endpoint issue in E2E
in toolbox mon endpoints are not
updated properly, this is causing an issue in E2E
this PR is a workaround to fix this issue.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2019-09-04 11:12:07 +00:00
Madhu Rajanna
677be13b11
Merge pull request #583 from nixpanic/mergify/dismiss_reviews
mergify: Remove approved review when a PR changes
2019-09-04 15:14:25 +05:30