Executing Commands in Go

The ability to execute external commands from within an application is something I often feel is a bit hackish and I haven’t yet discovered a language that handles it as well as I would like. That was until I learned to love how Go tackles the challenge. This post will show you how to make the most of os/exec .

I tend to feel that there are three different types of command execution within applications.

Plain output. For when you always expect the command to execute but all you want from it is the output.

Exit codes. These perform some cleanup, setup or check and you want to discard any output but assert the exit code.

Long running processes. This case is rarer but there are times when I want to spawn sub-processes. Maybe even starting up another server to proxy to for instance.

So how do I handle these in Go? The magic all comes from the wonderful os/exec which is part of the Go standard library. Using this library, commands become a first class citizen within your application.

For all examples I am only going to include the primary logic but I will keep a link to a full copy in the Go Playgroud. The copies won’t execute within the Playground due to the restrictions Google have in place but they should run fine locally.

I have defined the following functions to keep the examples short:

func printCommand ( cmd * exec . Cmd ) { fmt . Printf ( "==> Executing: %s

" , strings . Join ( cmd . Args , " " )) } func printError ( err error ) { if err != nil { os . Stderr . WriteString ( fmt . Sprintf ( "==> Error: %s

" , err . Error ())) } } func printOutput ( outs [] byte ) { if len ( outs ) > 0 { fmt . Printf ( "==> Output: %s

" , string ( outs )) } }

Collecting output

The first and most obvious use is to collect output from an external command. An easy way to do that is to use the CombinedOutput function.

// Create an *exec.Cmd cmd := exec . Command ( "echo" , "Called from Go!" ) // Combine stdout and stderr printCommand ( cmd ) output , err := cmd . CombinedOutput () printError ( err ) printOutput ( output ) // => go version go1.3 darwin/amd64 // http : // play . golang . org / p /- 7 PWDpt6zS

This works well if you also want to check for any error messages output but if you want finer control over the output of a command then we can route it into different buffers giving us control over both standard output and standard error.

// Create an *exec.Cmd cmd := exec . Command ( "go" , "version" ) // Stdout buffer cmdOutput := & bytes . Buffer {} // Attach buffer to command cmd . Stdout = cmdOutput // Execute command printCommand ( cmd ) err := cmd . Run () // will wait for command to return printError ( err ) // Only output the commands stdout printOutput ( cmdOutput . Bytes ()) // => go version go1.3 darwin/amd64 // http : // play . golang . org / p / _6xke11GMp

In the above example we manually connect our own buffer to capture the commands stdout stream. We can do the same for stderr and even stdin so long as it adheres to the io.Reader interface.

So far we’ve seen how easy it is to capture command output across multiple file descriptors. It’s more verbose then other languages but it gives us lots of flexibility.

Exit codes

Retrieving the exit code of a command is easy with Go. You may have already noticed in the previous examples that when executing the command Go can return an error. These errors occur if there is an issue with IO or of the command doesn’t return a successful exit code (0).

The following code will output the exit code of a command.

cmd := exec . Command ( "ls" , "/imaginary/directory" ) var waitStatus syscall . WaitStatus if err := cmd . Run (); err != nil { printError ( err ) // Did the command fail because of an unsuccessful exit code if exitError , ok := err .( * exec . ExitError ); ok { waitStatus = exitError . Sys ().( syscall . WaitStatus ) printOutput ([] byte ( fmt . Sprintf ( "%d" , waitStatus . ExitStatus ()))) } } else { // Command was successful waitStatus = cmd . ProcessState . Sys ().( syscall . WaitStatus ) printOutput ([] byte ( fmt . Sprintf ( "%d" , waitStatus . ExitStatus ()))) } // http : // play . golang . org / p / m2A17UWSOL

The above code looks more complicated but in traditional Go fashion it handles many situations.

Create an *exec.Cmd

Execute and test for any errors

If we received an error then check it was because of the commands exit

If no error then check the commands *os.ProcessState for the exit code (pretty much guaranteed to be 0 but in here for completion)

A caveat to take note of is that if Go failed to locate the command in your $PATH then it won’t ever execute the command and thus you will have no exit code. This is why it is important to assert the type of error returned.

Long running processes

All our above examples are synchronous. They wait for the command to complete before continuing the execution of our application. If we wanted to execute a long running task however it is likely that we want it to happen asynchronously. Once again this is trivially easy.

cmd := exec . Command ( "cat" , "/dev/random" ) randomBytes := & bytes . Buffer {} cmd . Stdout = randomBytes // Start command asynchronously err := cmd . Start () printError ( err ) // Create a ticker that outputs elapsed time ticker := time . NewTicker ( time . Second ) go func ( ticker * time . Ticker ) { now := time . Now () for _ = range ticker . C { printOutput ( [] byte ( fmt . Sprintf ( "%s" , time . Since ( now ))), ) } }( ticker ) // Create a timer that will kill the process timer := time . NewTimer ( time . Second * 4 ) go func ( timer * time . Timer , ticker * time . Ticker , cmd * exec . Cmd ) { for _ = range timer . C { err := cmd . Process . Signal ( os . Kill ) printError ( err ) ticker . Stop () } }( timer , ticker , cmd ) // Only proceed once the process has finished cmd . Wait () printOutput ( [] byte ( fmt . Sprintf ( "%d bytes generated!" , len ( randomBytes . Bytes ()))), ) // http : // play . golang . org / p / tQRk1xJOqW

Now that was a lot more work but it all makes good sense. I started a computationaly difficult task of generating a collection of random bytes. Next I start that command asynchronously and then begin a ticker to show the elapsed time and a timer to kill the process after 4 seconds. Once the process has been killed then we output the total number of generated bytes.

A small disclaimer. I don’t claim that this is the best way to do this but it demonstrates asynchronous commands and the ability to send signals to the process within our application.

This has been quite a technical post but we have covered lots of ground with the flexibility of os/exec . I have tested all the examples on my Mac using go1.3. Any suggestions/improvements are welcomed.

I hope I’ve been able to get across why I’m beginning to really enjoy working with Go.