Make Docs Into Code

20 August 2017

At clippy, we take onboarding new developers seriously. Just like documentation. And we always strive to improve our process (unless that keeps us from coding on clippy, which we love, as we wouldn’t be doing it otherwise in the first place).

So since documentation has come up during recent RustConf, and Manish has already given a talk about being welcoming to contributors, but alas he forgot to mention the thing I’m writing about now, I want to focus on a small thing that brings us a lot of goodness for a small code investment, and that is our update_wiki.py script.

Just to get it out of the way, this script is written in Python, because it’s well suited for the task and readily available for most of our developers. Now let’s look into how it came to be.

When we started clippy, we had very little in the way of documentation – at most a sentence describing what a lint should do and a bit about installation and usage. Seeing much better documentation from similar projects for other languages, we first set up a project wiki which I filled with information about the lints following a template I created that had roughly the following points:

What the lint searches for

Why this is bad

Known problems

Example

Default lint level

Lesson learned: Templates are awesome for documenting similar things.

Unfortunately, this was quite time-consuming. And it didn’t show up on my github timeline. Also the docs went out of date all the time, whenever a new lint was added or a bug was fixed. This was unsustainable.

So we came to the conclusion that we need to update the docs with the code. The only way to do this is to make the docs code. We needed to incorporate our docs into the code and ask contributors to keep it up to date.

Georg Brandl started this with a script that ran through our lint definitions (which use a format prescribed by rustc, and was easy to coopt here) and updated our README.md (It also updates the version information in the installation instructions and the libs.rs lint list.

Lesson learned: Automate away the tedium to remove boring busywork and ensure unrelenting accuracy.

Next, I used parts of his script to write a update_wiki.py script that would look for doc comments on the lint definitions (that were not interesting to rustdoc anyway) and create the wiki documentation we had. It also warned on lints that had no documentation on the wiki. I copied all our wiki docs into the code to start the transition.

This had two profound effects:

First, our docs were now updated with the code. Timeliness improved drastically, and has mostly stabilized. This was great for our users, because they no longer had to cope with stale docs (mostly).

Second, and in my opinion more important, doc contributions became code contributions. This means that doc changes use the same PR process as all other contributions, get the same review, mentoring and recognition as everything else. Also doc changes are often bundled with other code changes, which makes them even easier to review.

We have since moved away from the wiki and use a custom page made by Pascal Hertleif that allows you to browse the docs for all clippy versions.

Lesson learned: Rustdoc already makes docs code. Whenever you have (or need) docs that don’t fit its niche, seek out other ways to make the docs part of your code.