Often Go is chosen for a project because of its concurrency features. The Go team have gone to great lengths to make concurrency in Go cheap (in terms of hardware resources) and performant, however it is possible to use Go’s concurrency features to write code which is neither performent or reliable. With the time I have left I want to leave you with some advice for avoid some of the pitfalls that come with Go’s concurrency features.

Go features first class support for concurrency with channels, and the select and go statements. If you’ve learnt Go formally from a book or training course, you might have noticed that the concurrency section is always one of the last you’ll cover. This workshop is no different, I have chosen to cover concurrency last, as if it is somehow additional to the regular the skills a Go programmer should master.

There is a dichotomy here; Go’s headline feature is our simple, lightweight concurrency model. As a product, our language almost sells itself on this on feature alone. On the other hand, there is a narrative that concurrency isn’t actually that easy to use, otherwise authors wouldn’t make it the last chapter in their book and we wouldn’t look back on our formative efforts with regret.

This often eliminates a lot of state tracking and channel manipulation required to plumb a result back from a goroutine to its initiator.

So this is my first piece of advice: if your goroutine cannot make progress until it gets the result from another, oftentimes it is simpler to just do the work yourself rather than to delegate it.

I want to present to you another solution, one which has hopefully already occurred to you. Rather than run http.ListenAndServe in a goroutine, leaving us with the problem of what to do with the main goroutine, simply run http.ListenAndServe on the main goroutine itself.

An empty select statement will block forever. This is a useful property because now we’re not spinning a whole CPU just to call runtime.GoSched() . However, we’re only treating the symptom, not the cause.

Now, if you’re a little more experienced with go, you might instead write something like this.

This might look silly, but it’s a common common solution I see in the wild. It’s symptomatic of not understanding the underlying problem.

As the Go runtime is mostly cooperatively scheduled, this program is going to spin fruitlessly on a single CPU, and may eventually end up live-locked.

The program does what we intended, it serves a simple web server. However it also does something else at the same time, it wastes CPU in an infinite loop. This is because the for{} on the last line of main is going to block the main goroutine because it doesn’t do any IO, wait on a lock, send or receive on a channel, or otherwise communicate with the scheduler.

The solution to the problems of both implementations is to use a callback, a function that is called in the context of each directory entry as it is executed.

The caller must continue to read from the channel until it is closed because that is the only way the caller can know that the goroutine which was started to fill the channel has stopped. This is a serious limitation on the use of ListDirectory , the caller has to spend time reading from the channel even though it may have received the answer it wanted. It is probably more efficient in terms of memory usage for medium to large directories, but this method is no faster than the original slice based method.

By using a closed channel as the signal that there are no more items to process there is no way for ListDirectory to tell the caller that the set of items returned over the channel is incomplete because an error was encountered partway through. There is no way for the caller to tell the difference between an empty directory and an error to read from the directory entirely. Both result in a channel returned from ListDirectory which appears to be closed immediately.

Its not necessary for the second version to actually use a Go routine; it could allocate a channel sufficient to hold all the directory entries without blocking, fill the channel, close it, then return the channel to the caller. But this is unlikely, as this would have the same problems with consuming a large amount of memory to buffer all the results in a channel.

Lets look at the second example. This is a little more Go like, ListDirectory returns a channel over which directory entries will be passed. When the channel is closed, that is your indication that there are no more directory entries. As the population of the channel happens after ListDirectory returns, ListDirectory is probably starting a goroutine to populate the channel.

Firstly, the obvious differences; the first example reads a directory into a slice then returns the whole slice, or an error if something went wrong. This happens synchronously, the caller of ListDirectory blocks until all directory entries have been read. Depending on how large the directory, this could take a long time, and could potentially allocate a lot of memory building up the slide of directory entry names.

The previous example showed using a goroutine when one wasn’t really necessary. But one of the driving reasons for using Go is the first class concurrency features the language offers. Indeed there are many instances where you want to exploit the parallelism available in your hardware. To do so, you must use goroutines.

This simple application serves http traffic on two different ports, port 8080 for application traffic and port 8001 for access to the /debug/pprof endpoint.

package main import ( "fmt" "net/http" _ "net/http/pprof" ) func main() { mux := http.NewServeMux() mux.HandleFunc("/", func(resp http.ResponseWriter, req *http.Request) { fmt.Fprintln(resp, "Hello, QCon!") }) go http.ListenAndServe("127.0.0.1:8001", http.DefaultServeMux) // debug http.ListenAndServe("0.0.0.0:8080", mux) // app traffic }

Although this program isn’t very complicated, it represents the basis of a real application.

There are a few problems with the application as it stands which will reveal themselves as the application grows, so lets address a few of them now.

func serveApp() { mux := http.NewServeMux() mux.HandleFunc("/", func(resp http.ResponseWriter, req *http.Request) { fmt.Fprintln(resp, "Hello, QCon!") }) http.ListenAndServe("0.0.0.0:8080", mux) } func serveDebug() { http.ListenAndServe("127.0.0.1:8001", http.DefaultServeMux) } func main() { go serveDebug() serveApp() }

By breaking the serveApp and serveDebug handlers out into their own functions we’ve decoupled them from main.main . We’ve also followed the advice from above and make sure that serveApp and serveDebug leave their concurrency to the caller.

But there are some operability problems with this program. If serveApp returns then main.main will return causing the program to shutdown and be restarted by whatever process manager you’re using.

Tip Just as functions in Go leave concurrency to the caller, applications should leave the job of monitoring their status and restarting them if they fail to the program that invoked them. Do not make your applications responsible for restarting themselves, this is a procedure best handled from outside the application.

However, serveDebug is run in a separate goroutine and if it returns just that goroutine will exit while the rest of the program continues on. Your operations staff will not be happy to find that they cannot get the statistics out of your application when they want too because the /debug handler stopped working a long time ago.

What we want to ensure is that if any of the goroutines responsible for serving this application stop, we shut down the application.

func serveApp() { mux := http.NewServeMux() mux.HandleFunc("/", func(resp http.ResponseWriter, req *http.Request) { fmt.Fprintln(resp, "Hello, QCon!") }) if err := http.ListenAndServe("0.0.0.0:8080", mux); err != nil { log.Fatal(err) } } func serveDebug() { if err := http.ListenAndServe("127.0.0.1:8001", http.DefaultServeMux); err != nil { log.Fatal(err) } } func main() { go serveDebug() go serveApp() select {} }

Now serverApp and serveDebug check the error returned from ListenAndServe and call log.Fatal if required. Because both handlers are running in goroutines, we park the main goroutine in a select{} .

This approach has a number of problems:

If ListenAndServer returns with a nil error, log.Fatal won’t be called and the HTTP service on that port will shut down without stopping the application. log.Fatal calls os.Exit which will unconditionally exit the program; defers won’t be called, other goroutines won’t be notified to shut down, the program will just stop. This makes it difficult to write tests for those functions.

Tip Only use log.Fatal from main.main or init functions.

What we’d really like is to pass any error that occurs back to the originator of the goroutine so that it can know why the goroutine stopped, can shut down the process cleanly.

func serveApp() error { mux := http.NewServeMux() mux.HandleFunc("/", func(resp http.ResponseWriter, req *http.Request) { fmt.Fprintln(resp, "Hello, QCon!") }) return http.ListenAndServe("0.0.0.0:8080", mux) } func serveDebug() error { return http.ListenAndServe("127.0.0.1:8001", http.DefaultServeMux) } func main() { done := make(chan error, 2) go func() { done <- serveDebug() }() go func() { done <- serveApp() }() for i := 0; i < cap(done); i++ { if err := <-done; err != nil { fmt.Println("error: %v", err) } } }

We can use a channel to collect the return status of the goroutine. The size of the channel is equal to the number of goroutines we want to manage so that sending to the done channel will not block, as this will block the shutdown the of goroutine, causing it to leak.

As there is no way to safely close the done channel we cannot use the for range idiom to loop of the channel until all goroutines have reported in, instead we loop for as many goroutines we started, which is equal to the capacity of the channel.

Now we have a way to wait for each goroutine to exit cleanly and log any error they encounter. All that is needed is a way to forward the shutdown signal from the first goroutine that exits to the others.

It turns out that asking a http.Server to shut down is a little involved, so I’ve spun that logic out into a helper function. The serve helper takes an address and http.Handler , similar to http.ListenAndServe , and also a stop channel which we use to trigger the Shutdown method.

func serve(addr string, handler http.Handler, stop <-chan struct{}) error { s := http.Server{ Addr: addr, Handler: handler, } go func() { <-stop // wait for stop signal s.Shutdown(context.Background()) }() return s.ListenAndServe() } func serveApp(stop <-chan struct{}) error { mux := http.NewServeMux() mux.HandleFunc("/", func(resp http.ResponseWriter, req *http.Request) { fmt.Fprintln(resp, "Hello, QCon!") }) return serve("0.0.0.0:8080", mux, stop) } func serveDebug(stop <-chan struct{}) error { return serve("127.0.0.1:8001", http.DefaultServeMux, stop) } func main() { done := make(chan error, 2) stop := make(chan struct{}) go func() { done <- serveDebug(stop) }() go func() { done <- serveApp(stop) }() var stopped bool for i := 0; i < cap(done); i++ { if err := <-done; err != nil { fmt.Println("error: %v", err) } if !stopped { stopped = true close(stop) } } }

Now, each time we receive a value on the done channel, we close the stop channel which causes all the goroutines waiting on that channel to shut down their http.Server . This in turn will cause all the remaining ListenAndServe goroutines to return. Once all the goroutines we started have stopped, main.main returns and the process stops cleanly.