If you've ever developed with Python, you might have already met the cheese shop. Because this is a central repository for Python libraries (eggs), its downtime can severely affect your operation. Maybe you're here because you know this already.

So here's the problem: you want to have a PyPi mirror in you local network, because the main index' downtime costs you time, money and nerves. As a nice side-effect, you'll no longer hit the internet every time you need to gather dependencies. Here's what you'll need: a server with roughly 30GB or more of free disk space and a web server on it. I'll only show an nginx config because I can't be bothered to use Apache nowadays.

Let's assume this server is reachable on your network as piepie . On this server you need to prepare a folder where the mirror is stored. I'll asume this is /mnt/pypi .

The software which in my opinion is the most reliable to do the syncing is pep381client, which can be found on PyPi. Install it with pip (if you only have easy_install, then easy_install pip first):

$ pip install pep381client

Now do the initial sync. Because this will take a while, consider starting this in a screen/dtach/tmux session:

$ pep381run /mnt/pypi

After the initial run, add a crontab entry for the sync command by calling crontab -e and then adding a line like this:

@daily pep381run -q /mnt/pypi

The -q stands for quiet operation.

Now you need to tell nginx to serve the right subfolder, so cd to /var/www and

/var/www $ ln -s /mnt/pypi/web/

This should make your PyPi mirror available at http://piepie/pypi . To test that it works, try to install an egg with pip like this:

$ pip install -i http://piepie/pypi/simple

Assuming that the syncing was done properly, this should fetch the egg from your local server. To make this index permanent, add this to your ~/.pip/pip.conf file:

[global] index-url = http://piepie/simple extra-index-url = http://pypi.python.org/simple use-mirrors = true mirrors = http://piepie/pypi http://b.pypi.python.org http://c.pypi.python.org http://d.pypi.python.org http://e.pypi.python.org http://f.pypi.python.org

This config file makes pip check your local network's mirror first, then the main PyPi index, followed by a set of mirrors.

Discuss on Reddit

Discuss on HackerNews