Easily build Python C extensions using wheels on multiple architectures (Linux, MacOS, Windows) from an existing PyPI package.

The Problem

If you’re developing a Python package and need to develop a C extension, it’s confusing to figure out how to bundle and distribute it on PyPI. If your build step requires numpy or another package to be installed before setup.py runs, it becomes messy to provide your users with a single familiar command that will install your package.

You might be the developer of that package or you might be trying to pre-compile C extensions for an existing package.

A Potential Solution

Python wheels are an official binary distribution mechanism that specifies which OS (e.g. ubuntu) and python version (2 and/or 3) the bundle is good for. When a wheel is available for a system that invokes “pip install”, the binary will be download and installed without an additional compilation step. This provides a seamless experience for pip users.

However, you need access to servers running the right architectures to build enough binaries to be reasonably confident that your user base will be accommodated.

Wheelie

I propose a service, Wheelie, that takes a source distribution from PyPI and produces wheels for you on a variety of architectures using cloud services. Similar capabilities are possible using Travis CI (Travis docs and sample implementation). However, that solution requires setting up Travis CI and a cloud bucket on S3 or Google Storage which can be intimidating.

It may also not work well with your workflow if you like to manually decide when to release new binaries or don’t want to use a git release branch (maybe you use tags).

To use Wheelie, you’ll first release your source distribution on PyPI using setuptools and twine. Then you’ll go to the Wheelie website, log in, type in the name of your PyPI package, and click build. You’ll get an email when the build completes, can download the finished binaries, and release them using twine.

Fee

There will be some kind of small fee associated with the service. It’s necessary to power the servers and compensate my time. The first few builds will probably be free assuming that a large enough fraction of users are repeats to make the numbers work.

Interested?

Please tweet @willsilversmith or like this article. I’m attempting to gauge interest before investing too much time. You can email me directly at william dot silversmith plus wheelie at gmail dot com.

Thanks,

Will Silversmith

Software Developer