Commit Graph

3264 Commits

Author SHA1 Message Date
Madhu Rajanna
0bd45a5eb1 cephfs: use string const for cli error comparison
we should not return the CLI errors in GRPC errors
we need to return proper readable error messages
to the user for better understanding and better
debugging.

updates #1242

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-19 17:09:12 +00:00
Yug
78c1e781be doc: Add steps to set up git hook
Provide steps to install and set up
git hook locally.

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-08-19 16:01:16 +00:00
Yug
6287f6c6ec ci: Add pre-commit hook to catch issues locally
The Git hooks force the developers to satisty
certain rules before commiting changes(locally).

Following checks will be run before allowing developers
to commit changes:
1. Check if the commit message has a valid sign-off.
2. Catch gofmt issues, if any.
3. Verify syntax of yaml and json files.(if present)
4. Makes sure that files end in a new line.
5. Trims trailing whitespace.
6. Run commitlint check on the commit message.

The developers will not be allowed to commit changes
without satisfying the above mentioned rules; This will
inturn help in avoiding failures on CI.

Signed-off-by: Yug <yuggupta27@gmail.com>
2020-08-19 16:01:16 +00:00
Madhu Rajanna
27d2cc644d util: replace klog with util logger in liveness.go
replace klog with util logger in liveness.go

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-19 14:38:12 +00:00
Madhu Rajanna
ed289fea5e journal: replace klog with util logger in voljournal.go
journal: replace klog with util logger in voljournal.go

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-19 14:38:12 +00:00
Madhu Rajanna
1a9cd23f64 journal: replace klog with util logger in omap.go
replace klog with util logger in omap.go

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-19 14:38:12 +00:00
Madhu Rajanna
7c9fee17af rbd: replace klog with util logger in snapshot.go
rbd: replace klog with util logger in snapshot.go

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-19 14:38:12 +00:00
Madhu Rajanna
9305e27797 rbd: replace klog with util logger in rbd_util.go
rbd: replace klog with util logger in rbd_util.go

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-19 14:38:12 +00:00
Madhu Rajanna
90cc5a4fd0 rbd: replace klog with util logger in rbd_journal.go
replace klog with util logger in rbd_journal.go

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-19 14:38:12 +00:00
Madhu Rajanna
9512aa6206 rbd: replace klog with util logger in rbd_attach.go
replace klog with util logger in rbd_attach.go

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-19 14:38:12 +00:00
Madhu Rajanna
e768c0dfc0 rbd: replace klog with util logger in nodeserver.go
replace klog with util logger in nodeserver.go

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-19 14:38:12 +00:00
Madhu Rajanna
883485b082 rbd: replace klog with util logger in driver.go
replace klog with util logger in driver.go

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-19 14:38:12 +00:00
Madhu Rajanna
c4f30c23c1 rbd: replace klog with util logger in clone.go
replace klog with util logger in clone.go

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-19 14:38:12 +00:00
Madhu Rajanna
d8ca56fbe9 rbd: replace klog with util in controllerserver.go
replace klog with util in controllerserver.go

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-19 14:38:12 +00:00
Humble Chirammal
1f5b84745f rbd: Bail out from nodeexpansion if its block mode pvc
At CSI spec < 1.2.0, there was no volumecapability in the
expand request. However its available from v1.2+ which allows
us to declare the node operations based on the volume mode.

Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-19 12:34:20 +00:00
Prasanna Kumar Kalever
e9d5e731dc build: change default to use podman
With '# make image-cephcsi GOARCH=amd64'
On Fedora observed errors like:

 ---> Running in 3e2dbf48ebc6
OCI runtime create failed: this version of runc doesn't work on cgroups
v2: unknown
make: *** [Makefile:175: image-cephcsi] Error 1

hence it is recommended to switch to podman if available.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2020-08-19 11:52:10 +00:00
Madhu Rajanna
52243d0140 ci: error out for invalid component in commit header
if the commit header is not present in the list of
the component we maintain. the commitlint check should
show error instead of showing the warning.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-19 11:08:31 +00:00
Niels de Vos
fb9ae06b1f ci: mark .commitlintrc.yml as doc-only
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-08-19 09:58:34 +00:00
Madhu Rajanna
cc0f0b8a6a deploy: remove unnecessary aggregate clusterroles
The aggregate clusterrole were designed for the scenario where
the rules are not completely owned by one component.
the aggregate rules can be removed and simplify
certain issues around upgrades.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-19 09:30:17 +00:00
Niels de Vos
b66390ee06 ci: do not require commitlint before merging
The GitHub commitlint App does not get triggered for new or updated PRs
since the last days. It is possible to manually start a test in the
CentOS CI for this with `/test commitlint`, but Travis CI already
includes a test (as part of "static-checks") too. There is no need to
run the tests 2x, so remove the status check from the Mergify
configuration.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-08-19 13:45:40 +05:30
Prasanna Kumar Kalever
404ee73dcd cleanup: fix cmd in container img building errmsg
Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2020-08-19 07:06:28 +00:00
Niels de Vos
6d5456cc77 ci: use FETCH_HEAD like "commitlint --from" does
After fetching the target branch for the PR, the GIT_SINCE ref in the
git repository may not be set (in CI environments). This causes 'git
rebase' to fail. Use FETCH_HEAD instead, so that the checked out PR gets
rebased correctly.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-08-18 14:34:08 +00:00
Niels de Vos
104203ed03 ci: pass REBASE=1 to have commitlint rebase the branch
When Mergify adds a merge commit to the branch that is being tested with
commitlint, the tool tries to detect the most recent changes based on
the newly merged commit. This is for most PRs the master branch, and
that contains incorrect commit messages in the history. Because of this,
commitlint will fail.

By adding an option (REBASE=1) to the commitlint make target, CI jobs
can request a rebase so that the history of the PR becomes linear again
and commitlint should be able to detect only the new commits.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-08-18 14:34:08 +00:00
Niels de Vos
aca606ca42 build: add .test-container-id to .gitignore
Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-08-18 14:34:08 +00:00
Niels de Vos
84b7e7b11c build: store detected container-cmd to prevent rebuilds
The 'need-container-cmd' make target is marked as .PHONY which will
cause it to be run every time. That triggers a rebuild of the container
images, even when that is not required.

By removing the 'need-container-cmd' target from .PHONY, and storing the
contents of CONTAINER_CMD in .container-cmd, unneeded rebuilds are
prevented.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-08-18 14:34:08 +00:00
Madhu Rajanna
787ba69089 cephfs: fix lock release in volumeDelete
currently the lock is not released which is
taken on the request name. this is causing issues
when the subvolume is requested for delete.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-18 06:12:43 +00:00
Madhu Rajanna
31c7c95fdb cephfs: check error output contains Error: ENOENT
execCommandErr returns both error and stderror
message. checking strings.HasPrefix is not helpful
as the stderr will be the first string. its good
to do string comparison and find out that error
is volume not found error.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-18 06:12:43 +00:00
Niels de Vos
25899b0123 ci: do not constantly rebase PRs
It seems the 'rebase' action in Mergify causes each PR to be
automatically rebased once there is a change in the target branch. This
causes a very high load on the CI, which delays other PRs from getting
their test results.

Remove the 'rebase' action, as it does not work as intended. The
expectation was that '@mergifyio rebase' uses the configuration.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-08-17 16:23:12 +05:30
Niels de Vos
b881ef1eee ci: prevent Mergify from doing merge commits
Merge commits cause the CentOS CI commitlint job to fail. By configuring
Mergify to not do merge commits, but rebases before final testing, we
can use the CentOS CI commitlint job when the GitHub commitlint App does
not work.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-08-17 16:23:12 +05:30
yati1998
75d800c07e doc: documentation updates on capabilities
Documents exact capabilities a id requires against a
Ceph cluster for RBD and CephFS plugins.

Signed-off-by: yati1998 <ypadia@redhat.com>
2020-08-17 09:47:03 +00:00
Niels de Vos
aa5b53fcce ci: use ceph-csi-bot account for merging PRs
Currently the "@mergifyio rebase" commands uses a random person from the
GitHub organization that owns the repository. This is rather confusing
and ugly. With this change, alls rebase/merge actions are done with the
ceph-csi-bot account that also posts the result of jobs in the CentOS CI.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-08-17 08:30:26 +05:30
Madhu Rajanna
738b5a1f46 ci: add mergify rules for v3.1 branch
add mergify rules for backport and auto
merge of PR's for release-v3.1 branch.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-17 08:30:00 +05:30
Niels de Vos
c0fbaf4276 ci: only retry "kubectl create" if objects are missing
There can be spurious failures in the CI when running kubectl create. On
occasion, the command returns with an error, but the api-server did
receive and process the request. This causes a 2nd create action to fail
with messages like:

    cephcluster.ceph.rook.io/my-cluster created
    Error from server: error when creating "/tmp/tmp.Ur1ZPG85o9/cluster-test.yaml": etcdserver: request timed out
    Error from server (AlreadyExists): error when creating "/tmp/tmp.Ur1ZPG85o9/cluster-test.yaml": configmaps "rook-config-override" already exists
    Error from server (AlreadyExists): error when creating "/tmp/tmp.Ur1ZPG85o9/cluster-test.yaml": cephclusters.ceph.rook.io "my-cluster" already exists
    Error from server (AlreadyExists): error when creating "/tmp/tmp.Ur1ZPG85o9/cluster-test.yaml": configmaps "rook-config-override" already exists
    Error from server (AlreadyExists): error when creating "/tmp/tmp.Ur1ZPG85o9/cluster-test.yaml": cephclusters.ceph.rook.io "my-cluster" already exists
    Error from server (AlreadyExists): error when creating "/tmp/tmp.Ur1ZPG85o9/cluster-test.yaml": configmaps "rook-config-override" already exists
    Error from server (AlreadyExists): error when creating "/tmp/tmp.Ur1ZPG85o9/cluster-test.yaml": cephclusters.ceph.rook.io "my-cluster" already exists
    Error from server (AlreadyExists): error when creating "/tmp/tmp.Ur1ZPG85o9/cluster-test.yaml": configmaps "rook-config-override" already exists
    Error from server (AlreadyExists): error when creating "/tmp/tmp.Ur1ZPG85o9/cluster-test.yaml": cephclusters.ceph.rook.io "my-cluster" already exists

By handling the create action differently, and checking for the
AlreadyExists word in the stderr output, it is possible to detect
repeated creates that are not needed.

Signed-off-by: Niels de Vos <ndevos@redhat.com>
2020-08-13 16:28:28 +00:00
Madhu Rajanna
f11486f4b6 doc: add note for cephfs snapshot limitation
added a note for cephfs snapshot limitations

closes #1242

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 11:55:23 +00:00
Madhu Rajanna
fabe84d4fe doc: add upgrade documentation for v3.1.0
added upgrade documentation to upgrade from
v3.0.0 to v3.1.0

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 11:03:36 +00:00
Humble Chirammal
1b94ed2b37 doc: update cephfs snapshot, restore and clone support matrix
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>
2020-08-13 10:12:14 +00:00
Madhu Rajanna
f566b0fc2d cephfs: replace Warningf with WarningLog in unmountVolume
replace Errorf with ErrorLog in unmountVolume
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
3fe3d25dda cephfs: replace Errorf with ErrorLogMsg in loadAvailableMounters
replace Errorf with ErrorLog in loadAvailableMounters
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
fbe090065d cephfs: replace Errorf with ErrorLog in purgeVolume
replace Errorf with ErrorLog in purgeVolume
for context based logging

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
5a1652e5e8 cephfs: replace Errorf with ErrorLog in resizeVolume
replace Errorf with ErrorLog in resizeVolume
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
a882c1194b cephfs: replace Errorf with ErrorLog in createVolume
replace Errorf with ErrorLog in createVolume
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
f3d9b8933b cephfs: replace Errorf with ErrorLog in getSubVolumeInfo
replace Errorf with ErrorLog in getSubVolumeInfo
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
2b7aade289 cephfs: replace Errorf with ErrorLog in getVolumeRootPathCeph
replace Errorf with ErrorLog in getVolumeRootPathCeph
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
d1ac832266 cephfs: replace Errorf with ErrorLog in genSnapFromOptions
replace Errorf with ErrorLog in genSnapFromOptions
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
e83a51c1e1 cephfs: replace Errorf with ErrorLog in parseTime
replace Errorf with ErrorLog in parseTime
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
e7a043271d cephfs: replace Errorf with ErrorLog in cloneSnapshot
replace Errorf with ErrorLog in cloneSnapshot
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
841a3713f5 cephfs: replace Errorf with ErrorLog in unprotectSnapshot
replace Errorf with ErrorLog in unprotectSnapshot
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
91811c18c5 cephfs: replace Errorf with ErrorLog in protectSnapshot
replace Errorf with ErrorLog in protectSnapshot
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
72dfe6a265 cephfs: replace Errorf with ErrorLog in getSnapshotInfo
replace Errorf with ErrorLog in getSnapshotInfo
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00
Madhu Rajanna
7e8cc41aca cephfs: replace Errorf with ErrorLog in deleteSnapshot
replace Errorf with ErrorLog in deleteSnapshot
for context based logging.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
2020-08-13 09:44:12 +00:00