ceph-csi/docs/development-guide.md

Development Guide

New to Go

Ceph-csi is written in Go and if you are new to the language, it is highly encouraged to:

• Take the A Tour of Go course.
• Set up Go development environment on your machine.
• Read Effective Go for best practices.

Development Workflow

Workspace and repository setup

• Download Go (>=1.13.x) and install it on your system.

• Setup the GOPATH environment.

• CGO_ENABLED is enabled by default, if CGO_ENABLED is set to 0 we need to set it to 1 as we need to build with go-ceph bindings.

• GO111MODULE is enabled by default, if GO111MODULE is set to off we need to set it to on as cephcsi uses go modules for dependency.

• Ceph-CSI uses the native Ceph libaries through the go-ceph package. It is required to install the Ceph C headers in order to compile Ceph-CSI. The packages are called libcephfs-devel, librados-devel and librbd-devel on many Linux distributions. See the go-ceph installaton instructions for more details.

• Run $ go get -d github.com/ceph/ceph-csi This will just download the source and not build it. The downloaded source will be at $GOPATH/src/github.com/ceph/ceph-csi

• Fork the ceph-csi repo on Github.

• Add your fork as a git remote: $ git remote add fork https://github.com/<your-github-username>/ceph-csi

• Set up a pre-commit hook to catch issues locally.

  $ pip install pre-commit==2.5.1

  $ pre-commit install

  See the pre-commit installation instructions for more details.

  Pre-commit will be now be triggered next time we commit changes. This will catch some trivial style nitpicks (if any), which will then need resolving. Once the warnings are resolved, the user will be allowed to proceed with the commit.

Editors: Our favorite editor is vim with the vim-go plugin, but there are many others like vscode

Building Ceph-CSI

To build ceph-csi locally run: $ make

To build ceph-csi in a container: $ make containerized-build

The built binary will be present under _output/ directory.

Running Ceph-CSI tests in a container

Once the changes to the sources compile, it is good practise to run the tests that validate the style and other basics of the source code. Execute the unit tests (in the *_test.go files) and check the formatting of YAML files, MarkDown documents and shell scripts:

$ make containerized-test

It is also possible to run only selected tests, these are the targets in the Makefile in the root of the project. For example, run the different static checks with:

$ make containerized-test TARGET=static-check

In addition to running tests locally, each Pull Request that is created will trigger Continous Integration tests that include the containerized-test, but also additional functionality tests that are defined under the e2e/ directory.

Code contribution workflow

ceph-csi repository currently follows GitHub's [Fork & Pull] (https://help.github.com/articles/about-pull-requests/) workflow for code contributions.

Please read the coding guidelines document before submitting a PR.

Certificate of Origin

By contributing to this project you agree to the Developer Certificate of Origin (DCO). This document was created by the Linux Kernel community and is a simple statement that you, as a contributor, have the legal right to make the contribution. See the DCO file for details.

Contributors sign-off that they adhere to these requirements by adding a Signed-off-by line to commit messages. For example:

subsystem: This is my commit message

More details on what this commit does

Signed-off-by: Random J Developer <random@developer.example.org>

If you have already made a commit and forgot to include the sign-off, you can amend your last commit to add the sign-off with the following command, which can then be force pushed.

git commit --amend -s

We use a DCO bot to enforce the DCO on each pull request and branch commits.

Commit Messages

We follow a rough convention for commit messages that is designed to answer two questions: what changed and why? The subject line should feature the what and the body of the commit should describe the why.

cephfs: update cephfs resize

use cephfs resize to resize subvolume

Signed-off-by: Random J Developer <random@developer.example.org>

The format can be described more formally as follows:

<component>: <subject of the change>
<BLANK LINE>
<paragraph(s) with reason/description>
<BLANK LINE>
<signed-off-by>

The component in the subject of the commit message can be one of the following:

• cephfs: bugs or enhancements related to CephFS
• rbd: bugs or enhancements related to RBD
• doc: documentation updates
• util: utilities shared between components use cephfs or rbd if the change is only relevant for one of the type of storage
• journal: any of the journalling functionalities
• helm: deployment changes for the Helm charts
• deploy: updates to Kubernetes templates for deploying components
• build: anything related to building Ceph-CSI, the executable or container images
• ci: changes related to the Continuous Integration, or testing
• e2e: end-to-end testing updates
• cleanup: general maintenance and cleanup changes
• revert: undo a commit that was merged by mistake, use of one of the other components is in most cases recommended

The first line is the subject and should be no longer than 70 characters, the second line is always blank, and other lines should be wrapped at 80 characters. This allows the message to be easier to read on GitHub as well as in various git tools.

Here is a short guide on how to work on a new patch. In this example, we will work on a patch called hellopatch:

• $ git checkout master
• $ git pull
• $ git checkout -b hellopatch

Do your work here and commit.

Run the test suite, which includes linting checks, static code check, and unit tests:

$ make test

Certain unit tests may require extended permissions or other external resources that are not available by default. To run these tests as well, export the environment variable CEPH_CSI_RUN_ALL_TESTS=1 before running the tests.

You will need to provide unit tests and functional tests for your changes wherever applicable.

Once you are ready to push, you will type the following:

$ git push fork hellopatch

Creating A Pull Request: When you are satisfied with your changes, you will then need to go to your repo in GitHub.com and create a pull request for your branch. Automated tests will be run against the pull request. Your pull request will be reviewed and merged.

If you are planning on making a large set of changes or a major architectural change it is often desirable to first build a consensus in an issue discussion and/or create an initial design doc PR. Once the design has been agreed upon one or more PRs implementing the plan can be made.

Review Process: Once your PR has been submitted for review the following criteria will need to be met before it will be merged:

• Each PR needs reviews accepting the change from at least two developers for merging.
• It is common to request reviews from those reviewers automatically suggested by GitHub.
• Each PR needs to have been open for at least 24 working hours to allow for community feedback.
• The 24 working hours counts hours occurring Mon-Fri in the local timezone of the submitter.
• Each PR must be fully updated to master and tests must have passed
• If the PR is having trivial changes or the reviewer is confident enough that PR doesn't need a second review, the reviewer can set ready-to-merge label on the PR. The bot will merge the PR if it's having one approval and the label ready-to-merge.

When the criteria are met, a project maintainer can merge your changes into the project's master branch.

Backport a Fix to a Release Branch

The flow for getting a fix into a release branch is:

1. Open a PR to merge the changes to master following the process outlined above.
2. Add the backport label to that PR such as backport-to-release-vX.Y.Z
3. After your PR is merged to master, the mergify bot will automatically open a PR with your commits backported to the release branch
4. If there are any conflicts you will need to resolve them by pulling the branch, resolving the conflicts and force push back the branch
5. After the CI is green, the bot will automatically merge the backport PR.