Image by thiagociel

This article will combine all the following topics into one awesome step by chaining writers:

how to create a tarball.

how to generate an md5 hash for a file.

how to upload a file to a server.

The Tools

3 standard libraries are all thats needed to accomplish everything. Each of these libraries have something in common...

compress/gzip:

func NewWriter(w io Writer ) * Writer NewWriter returns a new Writer. Writes to the returned writer are compressed and written to w.

archive/tar:

func NewWriter(w io Writer ) * Writer NewWriter creates a new Writer writing to w.

crypto/md5:



// Write (via the embedded io.Writer interface)...

io.Writer

...

} type Hash interface {// Write (via the embedded io.Writer interface)......

md5.New() returns a hash.Hash which implements an io.Writer interface.

Putting it all together

Because all of these happen to use or implement an io.Writer in one way or another, you can combine them in really cool ways.

The example below creates a tarball, an md5 hash of the tarball, and then uploads the tarball to a server all at the same time by chaining the writers:

Take a step back to realize what is going on here...

The tar io.Writer is writing to a gzip io.Writer , which is writing to an io.MultiWriter that, in turn, is simultaneously writing to the md5.Hash io.Writer and an io.Pipe() which has an io.PipeWriter piping everything into an http.Request via its io.PipeReader … wow!

Don’t skip this part!

It’s imperative that I point something out here that is an easy “gotcha” if you’re not careful.

Because the code is making use of an io.Pipe() , either the io.PipeWriter or io.PipeReader must be in a goroutine . If one of them is not then the code will deadlock while the writer is waiting to write to a reader that is waiting to read from it.

Also, the when-and-where of how each reader/writer is closed, is very significant. You have to ensure that you don’t close a writer before a reader is done reading from it, and vice-versa. If you don’t have a reader to write to, or a writer to read from… bad things happen.