The Problem by Example

Let’s say you want to want to talk to spacely-sprockets.com via their nice JSON REST API and view a list of available sprockets. In Go, you might do something like:

// error checking omitted for brevity

var sprockets SprocketsResponse

response, _ := http.Get("spacely-sprockets.com/api/sprockets")

buf, _ := ioutil.ReadAll(response.Body)

json.Unmarshal(buf, &sprockets)

You write your code (with proper error handling, please), compile, and run. Everything works great. Now, you take your API package and plug it into a web application. One page of your app shows a list of Spacely Sprockets inventory to users by making a call to the API.

Everything is going great until one day your app stops responding. You look in the logs, but there’s nothing to indicate a problem. You check your monitoring tools, but CPU, memory, and I/O all look reasonable leading up to the outage. You spin up a sandbox and it seems to work fine. What gives?

Frustrated, you check Twitter and notice a tweet from the Spacely Sprockets dev team saying that they experienced a brief outage, but that everything is now back to normal. You check their API status page and see that the outage began a few minutes before yours. That seems like an unlikely coincidence, but you can’t quite figure out how it’s related, since your API code handles errors gracefully. You’re still no closer to figuring out the issue.