This is an overview of how I believe a real software project lifecycle can and should be managed when the tools at your disposal include a GitHub repository and a CI server. The explanation favors Jenkins but shouldn’t be exclusive to it.

It was originally written for work to finally get down in writing my opinions on project code, build and release management for the project I’ve suddenly started leading. I’m sharing it here because I believe these guidelines to be more generally applicable to any project that actually makes releases and has more than a single team member.

In an effort to reduce the number of pain and contention points during development as well as being able to maintain a high velocity without reducing the ability to experiment the following process should be applied to development of a given project. This applies for any developer providing code to the project: internal, external, senior or junior.

The process is always up for discussion and is not a dictated commandment, but a description of what seems to be a good idea over time. If in the process of applying the process something becomes clearly a bad idea we should try to figure out why and how to prevent it rather than pushing harder on enforcing process. Don’t be that guy, or that other guy and we’ll be ok.

Overview

The following assumptions should hold true at all times for the state of the repository and supporting tools.

Production code is always a git-tag ‘ed promoted Jenkins build

Any code running in the production environment must have arrived there from a successful build from Jenkins that has been promoted to production using the Promoted Builds Jenkins plugin. Evidence for exception to this principle should be of similar quality as that one would expect for a serial murder defense.

origin/master is always in good shape

The upstream master branch should reflect the latest, well-tested, peer-reviewed state of the repository. It doesn’t have to always reflect the tree in production, but anything in the master branch should certainly be destined there. The features should be well tested and serve as a reliable starting point for new work.

Any merge that causes master to fail the test suite should be reverted immediately to the last passing state and work to resolve the problem should happen elsewhere.

Every line of code is peer reviewed

Nothing should ever get merged into a mainline branch of the repo until at least two people have seen the code: The original author and a team member.

Any code that wants to be included in the mainline of the project should be introduced to the project by the author as a pull request. It doesn’t matter if the request comes from a branch within the repository or a fork of the same as long as the branch is rebased on the current branch the request targets. It should be the responsibility of the author of the branch to keep the branch rebased if its integration target moves during development. The final resulting merge should appear as forced merge commit on an otherwise fast-forward merge.

Pull requests should never be merged by the same person that authored them, and should always be done with the press of the green merge button on GitHub. If no one is available for review and :+1: please chase someone down rather than bending this rule. This one practice pays massive dividends.

Every pull request passes the test suite

Before any pull request to the project can be merged a comment should be added to the pull request with a link to the latest Jenkins build of the branch under review. Those merging the request should actively seek out such a link, even if one is known to exist and refuse to merge requests that omit it until one is available and present in the conversation thread.

This practice forces the author to assert through an unfeeling machine that the code presented for review and inclusion passes not only the full test suite of the application, but also the tests it included as part of development. The presence of this step also tends to eliminate seemingly simple changes with unforeseen consequences.

The convention the author wishes to introduce for including such comments in a pull request is:

Which renders using Emoji as a link to the build between two shiny sparkling thingies, which pretty much makes everyone happy. This also allows the shorthand conversational notation of “Sparkles” or “Sparkles build” for builds that pass the test suite, making everyone feel silly about themselves.

Everything that was ever a conversation is documented

If for any reason you need to offer an explanation, a walk through or a conversation about a topic relating to the development, deployment, operation or maintenance of the project you should instead of offering it directly take the time to write it down as a wiki page and link that instead.

This not only makes sure that the things we explain get written down for later, they can also serve as roadmaps or design documents for things still in development or earlier.

Since the internet is big and space is largely free documentation should not be reserved for special occasions. It should be written for design discussions, external and internal APIs, proposals, suggestions or even tangentially related ramblings. If it seems relevant, it should be linked together into groups of pages for easier navigation, like some sort of manual.

No new features without tests

If you write something that wasn’t there before, no one will believe you until you have tests proving that it works. When used in concert with the other guidelines this should also make sure that your feature works and does so well by the time it’s integrated.

Failure to do this should be easy to notice in peer review. If the feature is not only new, but also interesting it should also come with documentation.

On pull requests

Pull requests are the starting point of conversations about a set of changes. They are not demands, edicts, sticky notes or contracts. As such they can and should be opened whenever a conversation about a set of changes needs to take place. That is to say, they should not be a tool reserved for finished code waiting integration, they serve as an excellent platform to get feedback on code in progress or in question and should be used liberally.

Given that definition, the opening of a pull request should be given the same attention as a potential conversation with an audience. Don’t just accept the default copy inserted into the request based on your HEAD . Give the pull request a title that describes the code it carries and use the description field to explain the intent and approach of the changeset as well as any tradeoffs that may not be evident. Try to make the reviewers job easier rather than expecting them to divine details and direction of the change from the diff. It’ll save everyone time and sanity.

On Hotfixes

By the nature of the event, a Hotfix is something that is both urgent and completely unplanned so suggestion of a concrete process for dealing with them is quite ambitious and one won’t be attempted. Instead, these are some suggestions for how to find some generally relevant information and a best-case path for making an urgent code change, deploying it, and still managing to review and safely integrate it back into the codebase without generating a great deal of extra state, disruption and contention.

Determine which build is running in the affected environment by seeking the last promoted build in the CI environment. This should be linked with a specific build and an associated git commit. This would be a good place to branch from in the repository to implement the change with minimal impact.

You can do this by executing the following in the repository root.

$ git branch hotfix-description $COMMIT_SHA_FROM_CI_BUILD

The changes can then be made on the branch hotfix-description and pushed upstream as usual, triggering a CI build that can then be promoted to the required environment and deployed. If favorable, the same build can endure multiple, sequential promotions to be deployed first for internal testing in lower-risk sandbox or staging environments.

When the build carrying the hotfix is promoted and deployed to the desired environment and is tested to be a satisfactory solution to the hotfix condition, reintegration work can take place at a more relaxed pace without any incurred interruption to any other ongoing development work.

The hotfix-description branch can either be directly rebased on an upstream branch such as origin/master and replaced, left as-is, or copied to another upstream head. The resulting branch can then be submitted as a pull request for inclusion in the main line of code and a proper release can be made in the future, replacing the promoted hotfix build from earlier, but including the changes with proper review.

At least, as they say, that’s what is supposed to happen. In these cases, more so than most, defer to common sense, good judgement and experience.

TL;DR Checklist