Plumbum: Shell Combinators and More¶ Ever wished the compactness of shell scripts be put into a real programming language? Say hello to Plumbum Shell Combinators. Plumbum (Latin for lead, which was used to create pipes back in the day) is a small yet feature-rich library for shell script-like programs in Python. The motto of the library is “Never write shell scripts again”, and thus it attempts to mimic the shell syntax (shell combinators) where it makes sense, while keeping it all Pythonic and cross-platform. Apart from shell-like syntax and handy shortcuts, the library provides local and remote command execution (over SSH), local and remote file-system paths, easy working-directory and environment manipulation, quick access to ANSI colors, and a programmatic Command-Line Interface (CLI) application toolkit. Now let’s see some code!

News¶ 2020.03.23 : Version 1.6.9 released with several Path fixes, final version to support Python 2.6.

: Version 1.6.9 released with several Path fixes, final version to support Python 2.6. 2019.10.30 : Version 1.6.8 released with local.cmd , a few command updates, Set improvements, and TypedEnv .

: Version 1.6.8 released with , a few command updates, improvements, and . 2018.08.10 : Version 1.6.7 released with several minor additions, mostly to CLI apps, and run_* modifiers added.

: Version 1.6.7 released with several minor additions, mostly to CLI apps, and modifiers added. 2018.02.12 : Version 1.6.6 released with one more critical bugfix for a error message regression in 1.6.5.

: Version 1.6.6 released with one more critical bugfix for a error message regression in 1.6.5. 2017.12.29 : Version 1.6.5 released with mostly bugfixes, including a critical one that could break pip installs on some platforms. English cli apps now load as fast as before the localization update.

: Version 1.6.5 released with mostly bugfixes, including a critical one that could break pip installs on some platforms. English cli apps now load as fast as before the localization update. 2017.11.27 : Version 1.6.4 released with new CLI localization support. Several bugfixes and better pathlib compatibility, along with better separation between Plumbum’s internal packages.

: Version 1.6.4 released with new CLI localization support. Several bugfixes and better pathlib compatibility, along with better separation between Plumbum’s internal packages. 2016.12.31 : Version 1.6.3 released to provide Python 3.6 compatibility. Mostly bugfixes, several smaller improvements to paths, and a provisional config parser added.

: Version 1.6.3 released to provide Python 3.6 compatibility. Mostly bugfixes, several smaller improvements to paths, and a provisional config parser added. 2016.12.3 : Version 1.6.2 is now available through conda-forge, as well.

: Version 1.6.2 is now available through conda-forge, as well. 2016.6.25 : Version 1.6.2 released. This is mostly a bug fix release, but a few new features are included. Modifiers allow some new arguments, and Progress is improved. Better support for SunOS and other OS’s.

: Version 1.6.2 released. This is mostly a bug fix release, but a few new features are included. Modifiers allow some new arguments, and is improved. Better support for SunOS and other OS’s. 2015.12.18 : Version 1.6.1 released. The release mostly contains smaller fixes for CLI, 2.6/3.5 support, and colors. PyTest is now used for tests, and Conda is supported.

: Version 1.6.1 released. The release mostly contains smaller fixes for CLI, 2.6/3.5 support, and colors. PyTest is now used for tests, and Conda is supported. 2015.10.16 : Version 1.6.0 released. Highlights include Python 3.5 compatibility, the plumbum.colors package, Path becoming a subclass of str and a host of bugfixes. Special thanks go to Henry for his efforts.

: Version 1.6.0 released. Highlights include Python 3.5 compatibility, the package, becoming a subclass of and a host of bugfixes. Special thanks go to Henry for his efforts. 2015.07.17 : Version 1.5.0 released. This release brings a host of bug fixes, code cleanups and some experimental new features (be sure to check the changelog). Also, say hi to Henry Schreiner, who has joined as a member of the project.

: Version 1.5.0 released. This release brings a host of bug fixes, code cleanups and some experimental new features (be sure to check the changelog). Also, say hi to Henry Schreiner, who has joined as a member of the project. Change Log

Quick reference guide

Cheat Sheet¶ Basics¶ >>> from plumbum import local >>> ls = local [ "ls" ] >>> ls LocalCommand(<LocalPath /bin/ls>) >>> ls () u'build.py

dist

docs

LICENSE

plumbum

README.rst

setup.py

tests

todo.txt

' >>> notepad = local [ "c: \\ windows \\ notepad.exe" ] >>> notepad () # Notepad window pops up u'' # Notepad window is closed by user, command returns Instead of writing xxx = local["xxx"] for every program you wish to use, you can also import commands: >>> from plumbum.cmd import grep , wc , cat , head >>> grep LocalCommand(<LocalPath /bin/grep>) See Local Commands. Piping¶ >>> chain = ls [ "-a" ] | grep [ "-v" , " \\ .py" ] | wc [ "-l" ] >>> print chain /bin/ls -a | /bin/grep -v '\.py' | /usr/bin/wc -l >>> chain () u'13

' See Pipelining. Redirection¶ >>> (( cat < "setup.py" ) | head [ "-n" , 4 ])() u'#!/usr/bin/env python

import os



try:

' >>> ( ls [ "-a" ] > "file.list" )() u'' >>> ( cat [ "file.list" ] | wc [ "-l" ])() u'17

' See Input/Output Redirection. Working-directory manipulation¶ >>> local . cwd <Workdir /home/tomer/workspace/plumbum> >>> with local . cwd ( local . cwd / "docs" ): ... chain () ... u'15

' See Paths and The Local Object. Foreground and background execution¶ >>> from plumbum import FG , BG >>> ( ls [ "-a" ] | grep [ " \\ .py" ]) & FG # The output is printed to stdout directly build.py .pydevproject setup.py >>> ( ls [ "-a" ] | grep [ " \\ .py" ]) & BG # The process runs "in the background" <Future ['/bin/grep', '\\.py'] (running)> See Background and Foreground. Command nesting¶ >>> from plumbum.cmd import sudo >>> print sudo [ ifconfig [ "-a" ]] /usr/bin/sudo /sbin/ifconfig -a >>> ( sudo [ ifconfig [ "-a" ]] | grep [ "-i" , "loop" ]) & FG lo Link encap:Local Loopback UP LOOPBACK RUNNING MTU:16436 Metric:1 See Command Nesting. Remote commands (over SSH)¶ Supports openSSH-compatible clients, PuTTY (on Windows) and Paramiko (a pure-Python implementation of SSH2) >>> from plumbum import SshMachine >>> remote = SshMachine ( "somehost" , user = "john" , keyfile = "/path/to/idrsa" ) >>> r_ls = remote [ "ls" ] >>> with remote . cwd ( "/lib" ): ... ( r_ls | grep [ "0.so.0" ])() ... u'libusb-1.0.so.0

libusb-1.0.so.0.0.0

' See Remote. CLI applications¶ import logging from plumbum import cli class MyCompiler ( cli . Application ): verbose = cli . Flag ([ "-v" , "--verbose" ], help = "Enable verbose mode" ) include_dirs = cli . SwitchAttr ( "-I" , list = True , help = "Specify include directories" ) @cli.switch ( "-loglevel" , int ) def set_log_level ( self , level ): """Sets the log-level of the logger""" logging . root . setLevel ( level ) def main ( self , * srcfiles ): print "Verbose:" , self . verbose print "Include dirs:" , self . include_dirs print "Compiling:" , srcfiles if __name__ == "__main__" : MyCompiler . run () Sample output¶ $ python simple_cli.py -v -I foo/bar -Ispam/eggs x.cpp y.cpp z.cpp Verbose: True Include dirs: ['foo/bar', 'spam/eggs'] Compiling: ('x.cpp', 'y.cpp', 'z.cpp') See Command-Line Interface (CLI). Colors and Styles¶ from plumbum import colors with colors . red : print ( "This library provides safe, flexible color access." ) print ( colors . bold | "(and styles in general)" , "are easy!" ) print ( "The simple 16 colors or" , colors . orchid & colors . underline | '256 named colors,' , colors . rgb ( 18 , 146 , 64 ) | "or full rgb colors" , 'can be used.' ) print ( "Unsafe " + colors . bg . dark_khaki + "color access" + colors . bg . reset + " is available too." ) Sample output¶ This library provides safe color access. Color (and styles in general) are easy! The simple 16 colors, 256 named colors, or full hex colors can be used. Unsafe color access is available too. See Colors.

Development and Installation¶ The library is developed on GitHub, and will happily accept patches from users. Please use the GitHub’s built-in issue tracker to report any problem you encounter or to request features. The library is released under the permissive MIT license. Requirements¶ Plumbum supports Python 2.6-3.7 and PyPy and is continually tested on Linux, Mac, and Windows machines through Travis CI and Appveyor. Any Unix-like machine should work fine out of the box, but on Windows, you’ll probably want to install a decent coreutils environment and add it to your PATH . I can recommend mingw (which comes bundled with Git for Windows), but cygwin should work too. If you only wish to use Plumbum as a Popen-replacement to run Windows programs, then there’s no need for the Unix tools. Note that for remote command execution, an openSSH-compatible client is required (also bundled with Git for Windows), and a bash -compatible shell and a coreutils environment is also expected on the host machine. Download¶ You can download the library from the Python Package Index (in a variety of formats), or run pip install plumbum directly. If you use Anaconda, you can also get it from the conda-forge channel with conda install -c conda-forge plumbum .

About¶ The original purpose of Plumbum was to enable local and remote program execution with ease, assuming nothing fancier than good-old SSH. On top of this, a file-system abstraction layer was devised, so that working with local and remote files would be seamless. I’ve toyed with this idea for some time now, but it wasn’t until I had to write build scripts for a project I’ve been working on that I decided I’ve had it with shell scripts and it’s time to make it happen. Plumbum was born from the scraps of the Path class, which I wrote for the aforementioned build system, and the SshContext and SshTunnel classes that I wrote for RPyC. When I combined the two with shell combinators (because shell scripts do have an edge there) the magic happened and here we are.