Being a scripting language, Python lends itself to quick-and-dirty one-offs and throwaway code. But of course Python is also a full-fledged programming language many folks depend on for shipping code. Despite this, I’ve seen a lot of otherwise professional Python code packaged in myriad hacky ways, requiring custom installation and usage steps, manual or weird means of dependency installation, and unusual or unreliable PYTHONPATH shenanigans. Fortunately, packaging Python applications and libraries is quite simple. In fact, any time I write Python these days I always make sure it’s properly packaged.

Prerequisites

Apart from Python, make sure you have virtualenv. Any time you’re doing Python work which requires any dependencies, use virtualenv.

For the duration of this article, we’ll be working on a Python package of our own making called cooperative_engineering . Let’s set up a few things before getting to the interesting stuff.

Project skeleton

In an empty directory of your choosing:

$ mkdir cooperative_engineering $ touch cooperative_engineering/ { __init__.py,main.py } $ touch setup.py $ virtualenv venv Using base prefix '/usr' New python executable in /home/wilsoniya/devel/python-packaging/venv/bin/python Installing setuptools, pip, wheel...done. $ . venv/bin/activate ( venv ) $

Great, so now our directory should look like this:

. ├── cooperative_engineering │ ├── __init__.py │ └── main.py ├── setup.py └── venv ├── ... ...

And finally, let’s throw a little something in main.py :

#!/usr/bin/env python3 def tautology (): """Always return ``True``.""" return True def main (): """System entry point.""" print ( 'The rain in Spain falls mainly on the plain.' ) if __name__ == '__main__' : main ()

setup.py

setup.py is where the action happens. It’s a regular Python module, but by convention it contains a call setuptools.setup() , which acts as a CLI for working with your project.

Basics

#!/usr/bin/env python3 """``cooperative_engineering`` is a package we're creating to demonstrate how to properly package Python software. """ from setuptools import setup setup ( name = 'cooperative_engineering' , version = '0.1.0' , long_description = __doc__ , packages = [ 'cooperative_engineering' ], )

The call to setup() is the interesting bit above. All of the configuration of your package happens in various keyword arguments to setup() . Some details:

name : any text that describes the package, but I prefer to use the same value as the intended package name (here, cooperative_engineering )

: any text that describes the package, but I prefer to use the same value as the intended package name (here, ) version : an opaque string describing the version of the package (but probably use semver)

: an opaque string describing the version of the package (but probably use semver) long_description : human readable text describing the package. Here __doc__ is a reference to the module docstring.

: human readable text describing the package. Here is a reference to the module docstring. packages : a list of the package names which will be part of the distributed package. For simple packages such as ours, this value will correspond to the name of top-level directory in which our Python code resides.

What benefit does this get us? It’s not much, but try running python setup.py sdist . This will generate dist/cooperative_engineering-0.1.0.tar.gz which is a complete distribution of our package. This tarball could be uploaded to PyPI or installed on another system.

Dependencies

So what if we need to pull in third-party libraries?

setup ( name = 'cooperative_engineering' , version = '0.1.0' , long_description = __doc__ , packages = [ 'cooperative_engineering' ], install_requires = [ 'leftpad==0.1.2' , ], )

install_requires takes a list of package-version str s to be installed along alongside the package. Some advice:

Never declare unbounded dependencies

declare unbounded dependencies For executable packages, pin dependencies to exact versions, e.g. leftpad==0.1.2

For library packages, bound dependencies between the version of the dependency used for development, and the next major version, e.g. leftpad>=0.1.2,<1.0.0

Do not include transitive dependencies, i.e., dependencies of packages on which your package depends. setuptools will automatically resolve and install transitive dependencies.

Executables

Now let’s say we want to give the user of the package an easy way to run some part of it:

setup ( name = 'cooperative_engineering' , version = '0.1.0' , long_description = __doc__ , packages = [ 'cooperative_engineering' ], install_requires = [ 'leftpad==0.1.2' , ], entry_points = { 'console_scripts' : [ 'run-ce=cooperative_engineering.main:main' , ], }, )

Let’s unpack the line 'run-ce=cooperative_engineering.main:main' :

run-ce : the name of the command to which our code will be bound

: the name of the command to which our code will be bound cooperative_engineering.main:main : the package path and name of the function to be executed when the aforementioned command is invoked. Here, the main function within the module cooperative_engineering.main is targeted.

After making the highlighted addition to setup.py , do:

# activate the virtualenv if not already active $ . venv/bin/activate ( venv ) $ python setup.py develop

The develop subcommand to setup.py installs the cooperative_engineering package into your virtualenv, but rather than copying the source files into site-packages , the package is linked to so that subsequent source modifications are always reflected.

Caveat Adding additional console_scripts items will require an additional call to python setup.py develop . Why? setuptools generates executable scripts for each item in console_scripts . Check out the contents of ./venv/bin/ to see for yourself.

Now try:

# activate the virtualenv if not already active $ . venv/bin/activate ( venv ) $ run-ce The rain in Spain falls mainly on the plain.

Nice!

Tests

Because you 100% have written tests for your package, you’ll want to have a convenient way to run them:

setup ( name = 'cooperative_engineering' , version = '0.1.0' , long_description = __doc__ , packages = [ 'cooperative_engineering' ], install_requires = [ 'leftpad==0.1.2' , ], tests_require = [ 'nose2==0.8.0' ], test_suite = 'nose2.collector.collector' , entry_points = { 'console_scripts' : [ 'run-ce=cooperative_engineering.main:main' , ], }, )

This adds the lovely nose2 package as a dependency which will only be installed when we ask setuptools to run tests; dependencies listed under tests_require are not installed when the package is installed under normal circumstances. Let’s add a test in ./cooperative_engineering/tests/test_main.py :

#!/usr/bin/env python from unittest import TestCase from cooperative_engineering.main import tautology class TestTautology ( TestCase ): def test_tautology ( self ): self . assertTrue ( tautology ())

And then let’s run the new test:

# activate the virtualenv if not already active $ . venv/bin/activate ( venv ) $ python setup.py test running test Searching for nose2 Best match: nose2 0 .8.0 Processing nose2-0.8.0-py3.7.egg Using /home/wilsoniya/devel/python-packaging/.eggs/nose2-0.8.0-py3.7.egg running egg_info writing cooperative_engineering.egg-info/PKG-INFO writing dependency_links to cooperative_engineering.egg-info/dependency_links.txt writing entry points to cooperative_engineering.egg-info/entry_points.txt writing requirements to cooperative_engineering.egg-info/requires.txt writing top-level names to cooperative_engineering.egg-info/top_level.txt reading manifest file 'cooperative_engineering.egg-info/SOURCES.txt' writing manifest file 'cooperative_engineering.egg-info/SOURCES.txt' running build_ext . ---------------------------------------------------------------------- Ran 1 test in 0 .001s OK

LGTM

TL;DR

Here is a good starting point for packaging Python code.

Further reading