Sphinx documentation on GitHub

Quickly publish documentation alongside your code by following this easy recipe

The Data Desk of­ten pub­lishes doc­u­ment­a­tion along­side its code on Git­Hub, a so­cial net­work­ing site where de­velopers share their work. Git­Hub’s main ser­vice — free for open-source pro­jects — is for man­aging source code, but it also provides tick­et track­ing, wi­kis and a simple file-host­ing ser­vice for pub­lish­ing HTML pages.

It’s that last trick that’s great for doc­u­ment­a­tion. The ser­vice is called Git­Hub Pages and it’s tailored to host sites gen­er­ated by Je­kyll, a frame­work for cre­at­ing stat­ic sites with the Ruby pro­gram­ming lan­guage. But, if you make the right moves, you can get it to pub­lish just about any HTML. Not only is it live on­line, but every change you make will be saved in the same ver­sion-con­trol sys­tem as the code.

But what to put up there? You could hand­craft your doc­u­ment­a­tion, writ­ing out every HTML tag in lov­ing longhand. Or you could be lazy like the Data Desk and use soft­ware that does the hard work for you.

The Data Desk uses Sphinx, a frame­work for writ­ing and or­gan­iz­ing doc­u­ment­a­tion writ­ten in the Py­thon pro­gram­ming lan­guage. You provide the text and it provides a com­plete base tem­plate and auto­mat­ic­ally gen­er­ates a table of con­tents, nav­ig­a­tion bar and links between dif­fer­ent pages. It even in­dexes your site and sets up a simple JavaS­cript-based search.

The text is ex­pec­ted in re­Struc­tured­Text markup and the base tem­plate can be themed to look however you like. The Data Desk’s style is based on the design of The Times’ Sunday news­pa­per and can be seen here, here, here, here and here. You can also see Sphinx in use on doc­u­ment­a­tion for Py­thon’s core, pip, Django and Flask.

It’s a bit of trick to get Sphinx to play nice with Git­Hub Pages, but it can be done. Here’s how.

Step by step

Git­Hub is powered by Git, the ver­sion con­trol sys­tem de­veloped for man­aging the Linux source code. Most everything you do on Git­hub is centered on Git-man­aged code re­pos­it­or­ies.

So that’s where we start. If you don’t have a Git­Hub ac­count already, set one up. Then cre­ate a new code re­pos­it­ory and fol­low the in­struc­tions to get it set up on your com­puter.

To start, your code is go­ing to go in the “mas­ter” branch cre­ated when Git first ini­tial­izes your re­pos­it­ory. Nor­mally, you cre­ate oth­er branches when you want to fork away and make changes in­de­pend­ent of what’s in the mas­ter branch.

With Git­Hub Pages, any­thing you com­mit to a branch titled gh-pages will in­stead be pub­lished at ht­tp://your_user­name.git­hub.com/your_re­pos­it­ory_­name/. So the first step to pub­lish­ing any­thing is to cre­ate that branch.

$ git branch gh-pages

Now jump in­to the new branch.

$ git checkout gh-pages

Be­fore we can start work, we need to clear out any­thing from the mas­ter branch and start fresh. To make that hap­pen, you have to run a few ob­scure Git com­mands. Here they are.

$ git symbolic-ref HEAD refs/heads/gh-pages $ rm .git/index $ git clean -fdx

Once that’s over, in­stall Sphinx from the PyPI pack­age re­pos­it­ory us­ing pip .

$ pip install sphinx

Once Sphinx is in­stalled, in­struct it to cre­ate a new doc­u­ment­a­tion set.

$ sphinx-quickstart

After you is­sue the com­mand, Sphinx will ask you a series of ques­tions that con­fig­ure your pro­ject. You can give the de­fault an­swer in every case ex­cept one. The second ques­tion it will ask is:

Separate source and build directories (y/N) [n]:

You must an­swer yes. That means you should type Y and then press enter on your key­board.

Next you have to modi­fy Sphinx’s Makefile so it will build your site in the root dir­ect­ory. Open up the Makefile and change the eighth line to:

BUILDDIR = ./

Then go down to the 39th line and change the html com­mand to look like this:

html: $(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR) @echo @echo "Build finished. The HTML pages are in $(BUILDDIR)/html."

To veri­fy it works, in­struct Sphinx to build your site for the first time. You will need to is­sue this com­mand every time you wish to up­date the files on your loc­al sys­tem.

$ make html

Fi­nally, you need to cre­ate an empty file in the root dir­ect­ory that lets Git­Hub know you aren’t us­ing Je­kyll to struc­ture your site.

$ touch .nojekyll

Now all that’s left is to com­mit your HTML and push it up to Git­Hub.

$ git add . $ git commit -am "First commit" $ git push origin gh-pages

Wait a few minutes and vis­it ht­tp://your_user­name.git­hub.com/your_re­pos­it­ory_­name/. There you should find a simple Sphinx site live on the Web. To up­date it, edit the files in the source dir­ect­ory, run make html again and push to Git­Hub.

Ty­ing it to­geth­er

The routine above will work, but it is tire­some to run through it each time you start a new re­pos­it­ory. You can auto­mate the pro­cess by chain­ing all of the com­mands to­geth­er and ex­ecut­ing them as a batch. The Data Desk does this us­ing Fab­ric, a tool for stream­lin­ing Py­thon tasks. It was de­signed for de­ploy­ing code to re­mote ma­chines, but it’s just as use­ful for run­ning com­mands on your loc­al com­puter.

If you don’t have Fab­ric in­stalled already, re­trieve it from PyPI.

$ pip install fabric

Once it’s ready, cre­ate a new file called fabfile.py at the root of your mas­ter branch and paste in the fol­low­ing code.

from fabric.api import * from fabric.contrib.console import confirm def create_sphinx_pages (): """ Create a new branch with Sphinx documentation ready to be published using GitHub's Pages system. Example usage: $ fab make_sphinx_branch Before you can publish your docs, you need to commit them to the repo. $ git add . $ git commit -am "First commit" Then publish the files by pushing them up to GitHub. $ git push origin gh-pages Then the docs will appear on GitHub at: http://<your_account_name>.github.com/<your_repo_name>/ """ # Create the new branch local ( "git branch gh-pages" ) # Move into it local ( "git checkout gh-pages" ) # Clear it out local ( "git symbolic-ref HEAD refs/heads/gh-pages" ) local ( "rm .git/index" ) local ( "git clean -fdx" ) # Install sphinx local ( "pip install sphinx" ) # Save the dependencies to the requirements file local ( "pip freeze > requirements.txt" ) # Warn the user of a quirk before configuring with Sphinx confirm ( """. ___ ___ _ ___ ___ _ /\ | | |_ |\ | | | / \ |\ | /--\ | | |_ | \| | _|_ \_/ | \| Sphinx is about to start configuring your project. You can accept the default settings it offers, EXCEPT ONE . The second question it will ask is: 'Separate source and build directories (y/N) [n]:' YOU MUST ANSWER YES . THAT MEANS YOU TYPE 'Y' AND PRESS ENTER . DO YOU UNDERSTAND ?""" ) # Start up a Sphinx project local ( "sphinx-quickstart" ) # Create the .nojekyll file GitHub requires local ( "touch .nojekyll" ) # Make the patches to Sphinx's Makefile we need local ( "echo '' >> Makefile" ) local ( "echo ' BUILDDIR = ./' >> Makefile" ) local ( "echo '' >> Makefile" ) local ( "echo 'html:' >> Makefile" ) local ( "echo ' \t $( SPHINXBUILD ) -b html $( ALLSPHINXOPTS ) $( BUILDDIR )' >> Makefile" ) local ( "echo ' \t @echo' >> Makefile" ) local ( "echo ' \t @echo \" Build finished. The HTML pages are in $( BUILDDIR ) \" ' >> Makefile" ) # Make the branch for the first time local ( "make html" )

Fab­ric does ex­actly the same thing we walked through above, but it does it with one simple com­mand.

$ fab create_sphinx_pages

And all that’s left is the hon­or of pub­lish­ing it for the first time.

$ git add . $ git commit -am "First commit" $ git push origin gh-pages

Much re­spect due

This post pulls to­geth­er ideas worked out else­where by Luca Sbar­della, Dami­en Lebrun, Mi­chael Jones and Tom Pre­ston-Wern­er, who all de­serve our thanks.