This tutorial combines two of my favorite things, the Go programming language and images of SpaceX rocket launches.

With Go rapidly picking up adoption in the developer community, its becoming one of the leading languages for building backend systems. Go’s performance is similar to Java and C++, yet it’s almost as easy to write as Python. Since its inception in 2009 by Google, Go has been adopted by Netflix, Stripe, Twitch, Digital Ocean and many others. Here at Stream we use Go & RocksDB to power news feeds for over 300 million end users.

In this tutorial, you’ll build a highly concurrent web server which reads SpaceX images from the Unsplash API and applies seam carving on those images. The pace is high and assumes you already know how to program. You’ll learn the basics of Go and we’ll also cover more advanced topics such as Goroutines and Channels.

Part 1 - Setup and Seam Carving

Step 1 - Install Go 1.11

Homebrew on macOS is the easiest way to install Go:

brew install go

Alternatively head over to the official Go download page and follow the install instructions for your system.

Be sure that you have installed Go 1.11 or later before continuing this guide. Once installed, check your version with

go version

Step 2 - A Simple Request Handler

Go v1.11 simplified the initial setup of a Go project. Older versions of Go required the GOPATH environment variable. With v1.11 you can follow the three steps below to get your server up and running:

A. Create a directory

Create an empty directory called rockets:

mkdir rockets cd rockets

B. Go mod init

Run the following command to start a new Go module:



go mod init github.com/GetStream/rockets-go-tutorial

This will create a file called go.mod in your directory. This file will contain your dependencies. It is similar to package.json in Node, or requirements.txt in Python. (Go doesn’t have a package repository

C. main.go

Create a file called main.go in the rockets directory with the following content. (The full path is rockets/main.go):

package main import ( "bytes" "fmt" "io" "log" "net/http" "github.com/esimov/caire" "github.com/pkg/errors" ) const ( IMAGE_URL string = "https://bit.ly/2QGPDkr" ) func ContentAwareResize(url string) ([]byte, error) { fmt.Printf("Download starting for url %s

", url) response, err := http.Get(url) if err != nil { return nil, errors.Wrap(err, "Failed to read the image") } defer response.Body.Close() converted := &bytes.Buffer{} fmt.Printf("Download complete %s", url) shrinkFactor := 30 fmt.Printf("Resize in progress %s, shrinking width by %d percent...

", url, shrinkFactor) p := &caire.Processor{ NewWidth: shrinkFactor, Percentage: true, } err = p.Process(response.Body, converted) if err != nil { return nil, errors.Wrap(err, "Failed to apply seam carving to the image") } fmt.Printf("Seam carving completed for %s

", url) return converted.Bytes(), nil } func main() { fmt.Println("Ready for liftoff! Checkout http://localhost:3000/occupymars") http.HandleFunc("/occupymars", func(w http.ResponseWriter, r *http.Request) { if r.URL.Query().Get("resize") > "" { resized, err := ContentAwareResize(IMAGE_URL) if err != nil { http.Error(w, err.Error(), http.StatusInternalServerError) return } w.Header().Set("Content-Type", "image/jpeg") io.Copy(w, bytes.NewReader(resized)) } else { fmt.Fprintf(w, " Original image:

Resize using Seam Carving", IMAGE_URL) } }) log.Fatal(http.ListenAndServe(":3000", nil)) }

You can run it like this:

go run main.go

After a moment of installing dependencies, you’ll be able to visit http://localhost:3000/occupymars

Click the “Resize using Seam Carving” link to start the resizing. The computation is quite heavy and may take a bit, so give it a few seconds.

Voila, one resized rocket picture:

Step 3 - Go’s Syntax

The first thing you’ll notice in the above example is that Go is very similar to other programming languages. Let’s quickly review the code in main.go:

A. Go is a statically typed language

func ContentAwareResize(url string) ([]byte, error) { ... }

The function takes a string as input and returns a slice of bytes and an error.

B. Go infers types

shrinkFactor := 30

Is short for:

var shrinkFactor int = 30

Here’s the syntax for the most commonly used types in Go:

message := "hello world" friends := []string{"john", "amy"} friends = append(friends, "jack") populationByCity := map[string]int{"Boulder": 108090, "Amsterdam": 821752} populationByCity["Palo Alto"] = 67024

LearnXinY is a great resource to quickly check the Go syntax.

C. Types are also inferred when calling functions

resized, err := ContentAwareResize(url)

Note how the function returns two elements which we use to initialize these new variables: resized and err.

D. Go & Unused variables

One of the quirks of Go is that unlike most other languages it will throw a syntax error if you leave a variable or import unused. If you add the following code after the shrinkFactor on line 29:

oldvariable := 20

And run main.go you’ll get the following error:

./main.go:30:12: oldvariable declared and not used

Most editors will help you clean up unused code, so in practice this is not that annoying. If you want to ignore one of the variables returned by a function you can do it like this:

_, err := http.Get(url)

The _ simply means discard the value.

E. Interfaces

On line 36 the p.Process function converts the image in response.Body and stores it in the converted variable. What’s interesting is the function definition of p.Process:

func (p *Processor) Process(r io.Reader, w io.Writer) error { … }

This function takes an io.Reader as the first argument and an io.Writer as the second. io.Reader and io.Writer are interfaces. Any type that implements the methods required by these interfaces can be passed to the function.

Note: One interesting fact about interfaces in Go is that types implement them implicitly. There is no explicit declaration of intent, no "implements" keyword. A type implements an interface by applying its methods.

(Fun read: This blogpost about Streaming IO in Go gives more details about the io.Reader and io.Writer interfaces)

F. Imports

import ( "bytes" "fmt" "io" "log" "net/http" "github.com/esimov/caire" "github.com/pkg/errors" )

The import syntax is easy to understand. Let’s refactor the ContentAwareResize function into a separate package to clean up our code.

Step 4 - Refactoring to a Package

A. Create a directory with the name “seam”:

mkdir seam

Your full path will look like rockets/seam.

Note: If you’re wondering why the folder is called seam it’s because this type of content aware image resizing is also typically called seam carving.

B. In the seam directory, create a file called “seam.go” with the following content:

package seam import ( "bytes" "fmt" "net/http" "github.com/esimov/caire" "github.com/pkg/errors" ) func ContentAwareResize(url string) ([]byte, error) { fmt.Printf("Download starting for url %s

", url) response, err := http.Get(url) if err != nil { return nil, errors.Wrap(err, "Failed to read the image") } defer response.Body.Close() converted := &bytes.Buffer{} fmt.Printf("Download complete %s

", url) shrinkFactor := 30 fmt.Printf("Resize in progress %s, shrinking width by %d percent...

", url, shrinkFactor) p := &caire.Processor{ NewWidth: shrinkFactor, Percentage: true, } err = p.Process(response.Body, converted) if err != nil { return nil, errors.Wrap(err, "Failed to apply seam carving to the image") } fmt.Printf("Seam carving completed for %s

", url) return converted.Bytes(), nil }

C. Open main.go and make the following changes:

Remove the import for "github.com/esimov/caire" Remove the import for "github.com/pkg/errors" Add the import for "github.com/GetStream/rockets-go-tutorial/seam" Update the function call in main.go from ContentAwareResize to seam.ContentAwareResize (Line 50) Remove the ContentAwareResize function from main.go (Line 18-43)

To clarify, before the changes the function call looked like this:

resized, err := ContentAwareResize(url)

After the changes you’re calling the method defined in the seam module like this:

resized, err := seam.ContentAwareResize(url)

D. See if it worked:

Start the server and see if it still works!

go run main.go

If everything went according to plan you should still be able to access at http://localhost:3000/occupymars

Note: Packages give you a nice way to structure your code. Capitalized functions are public and lowerCase functions are private.

If something went wrong with these steps you can also clone the Github repo for a working version:

Part 2 - Unsplash API, Channels and Concurrency

Step 5 - APIs, Structs and JSON

You’re already well on your way to learning Go. Easy right? Not bad for a language that’s 20-40 times faster than Python!

For this second example we’ll learn how to use read data from an API. We’ll use the excellent Unsplash photography API to search for more rocket pictures.

A. Create a folder called unsplash:

mkdir unsplash

The full path is rockets/unsplash.

B. Inside the unsplash directory create a file called unsplash.go with the following content:

package unsplash import ( "encoding/json" "fmt" "net/http" "github.com/pkg/errors" ) type APIResponse struct { Total int json:"total" TotalPages int json:"total_pages" Results []PictureResult json:"results" } type PictureResult struct { ID string json:"id" Width int json:"width" Height int json:"height" URLs map[string]string json:"urls" Resized string } type APIClient struct { // note how the lowercase accessToken is private accessToken string } func NewAPIClient(token string) APIClient { return APIClient{token} } func (c *APIClient) Search(query string) (*APIResponse, error) { url := fmt.Sprintf("https://api.unsplash.com/search/photos?page=1&query=%s&client_id=%s", query, c.accessToken) fmt.Println(url) resp, err := http.Get(url) if err != nil { return nil, errors.Wrap(err, "failed to read from Unsplash API") } defer resp.Body.Close() response := APIResponse{} err = json.NewDecoder(resp.Body).Decode(&response) if err != nil { return nil, errors.Wrap(err, "failed to parse JSON") } return &response, nil } func LoadRockets() (*APIResponse, error) { query := "spacex" client := NewAPIClient("c1f9504a548ec5ea75acf3a3919ceab1ab04d09b732a839f04ca0be74f6227a0") response, err := client.Search(query) return response, err }

C. Update rockets/main.go to match this content:

https://raw.githubusercontent.com/GetStream/rockets-go-tutorial/master/main.go

D. Create a file called rockets/spacex.html:

The content should match this:

{% for result in response.Results|slice:":8" %} {% if result.Resized %} {% endif %} {% endfor %}

E. Restart your Go server:

go run main.go

Assuming nothing went wrong you can now open up this URL

http://localhost:3000/spacex

You should see the latest images tagged with SpaceX on Unsplash:

Wicked!

Step 6 - Understand Those API Calls

Let’s open up unsplash.go and have a look at how the code works:

A. Making the request:

Go’s builtin HTTP library is pretty functional, here’s the GET request to the Unsplash API

resp, err := http.Get(url)

B. Structs and methods:

Go has objects but it’s different from traditional object oriented languages because it doesn’t support inheritance. Go relies on composition instead of inheritance to help you structure your code. Structs in Go are the closest you’ll come to classes in more object oriented languages. Let’s see how we’re using Structs to talk to the Unsplash API in unsplash.go:

type APIClient struct { // note how the lowercase accessToken is private accessToken string } func NewAPIClient(token string) APIClient { return APIClient{token} } func (c *APIClient) Search(query string) (*APIResponse, error) { ... return &response, nil }

The NewAPIClient creates a new APIClient struct. The Search method enables you to write code like:

client := NewAPIClient("accessToken") response, err := client.Search(query)

This blogpost explains the concept of composition nicely.

C. Parsing JSON:

Since Go is a statically typed language you’ll want to parse the JSON into a Struct. If you’re coming from a dynamic language this will be a little bit confusing at first. You’ll get the hang of it quickly. The Unsplash API returns the following JSON:

{ "total": 133, "total_pages": 7, "results": [ { "id": "eOLpJytrbsQ", "created_at": "2014-11-18T14:35:36-05:00", "urls": { "raw": "https://images.unsplash.com/photo-1416339306562-f3d12fefd36f", "full": "https://hd.unsplash.com/photo-1416339306562-f3d12fefd36f" }, } ] }

There are more fields in the JSON but I simplified it a bit for the example. Next we’ll want to parse the JSON into the following structs:

type APIResponse struct { Total int json:"total" TotalPages int json:"total_pages" Results []PictureResult json:"results" } type PictureResult struct { ID string json:"id" Width int json:"width" Height int json:"height" URLs map[string]string json:"urls" Resized string }

The struct definition specifies the mapping between the field name in the JSON and the struct:

TotalPages int json:"total_pages"

This means that when decoding JSON it will take the value from the total_page and set it to the TotalPages property.

To decode the JSON we use the following code:

response := APIResponse{} err = json.NewDecoder(resp.Body).Decode(&response) if err != nil { return nil, errors.Wrap(err, "failed to parse JSON") }

We create a new APIResponse struct and parse a pointer to this object in the Decode function.

The next section will discuss pointers in more detail.

D. The defer statement

One of the fairly unique concepts of Go is the defer statement. In unsplash.go line 41 you see the following defer statement:

defer resp.Body.Close()

Deferred function calls are pushed onto a stack. When a function returns, its deferred calls are executed in last-in-first-out order. So after the Search function finishes the defer statement will trigger and close the request body.