Testing Techniques Google I/O 2014 Andrew Gerrand

Video This talk was presented at golang-syd in July 2014. Watch the talk on YouTube 2

The basics 3

Testing Go code Go has a built-in testing framework. It is provided by the testing package and the go test command. Here is a complete test file that tests the strings.Index function: package strings_test import ( "strings" "testing" ) func TestIndex(t *testing.T) { const s, sep, want = "chicken", "ken", 4 got := strings.Index(s, sep) if got != want { t.Errorf("Index(%q,%q) = %v; want %v", s, sep, got, want) } } 4

Table-driven tests Go's struct literal syntax makes it easy to write table-driven tests: func TestIndex(t *testing.T) { var tests = []struct { s string sep string out int }{ {"", "", 0}, {"", "a", -1}, {"fo", "foo", -1}, {"foo", "foo", 0}, {"oofofoofooo", "f", 2}, // etc } for _, test := range tests { actual := strings.Index(test.s, test.sep) if actual != test.out { t.Errorf("Index(%q,%q) = %v; want %v", test.s, test.sep, actual, test.out) } } } 5

T The *testing.T argument is used for error reporting: t.Errorf("got bar = %v, want %v", got, want) t.Fatalf("Frobnicate(%v) returned error: %v", arg, err) t.Logf("iteration %v", i) And enabling parallel tests: t.Parallel() And controlling whether a test runs at all: if runtime.GOARCH == "arm" { t.Skip("this doesn't work on ARM") } 6

Running tests The go test command runs tests for the specified package.

(It defaults to the package in the current directory.) $ go test PASS $ go test -v === RUN TestIndex --- PASS: TestIndex (0.00 seconds) PASS To run the tests for all my projects: $ go test github.com/nf/... Or for the standard library: $ go test std 7

Test coverage The go tool can report test coverage statistics. $ go test -cover PASS coverage: 96.4% of statements ok strings 0.692s The go tool can generate coverage profiles that may be intepreted by the cover tool. $ go test -coverprofile=cover.out $ go tool cover -func=cover.out strings/reader.go: Len 66.7% strings/strings.go: TrimSuffix 100.0% ... many lines omitted ... strings/strings.go: Replace 100.0% strings/strings.go: EqualFold 100.0% total: (statements) 96.4% 8

Coverage visualization $ go tool cover -html=cover.out 9

Advanced techniques 10

An example program outyet is a web server that announces whether or not a particular Go version has been tagged. go get github.com/golang/example/outyet 11

Testing HTTP clients and servers The net/http/httptest package provides helpers for testing code that makes or serves HTTP requests. 12

httptest.Server An httptest.Server listens on a system-chosen port on the local loopback interface, for use in end-to-end HTTP tests. type Server struct { URL string // base URL of form http://ipaddr:port with no trailing slash Listener net.Listener // TLS is the optional TLS configuration, populated with a new config // after TLS is started. If set on an unstarted server before StartTLS // is called, existing fields are copied into the new config. TLS *tls.Config // Config may be changed after calling NewUnstartedServer and // before Start or StartTLS. Config *http.Server } func NewServer(handler http.Handler) *Server func (*Server) Close() error 13

httptest.Server in action This code sets up a temporary HTTP server that serves a simple "Hello" response. // +build ignore,OMIT package main import ( "fmt" "io/ioutil" "log" "net/http" "net/http/httptest" ) func main() { ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { fmt.Fprintln(w, "Hello, client") })) defer ts.Close() res, err := http.Get(ts.URL) if err != nil { log.Fatal(err) } greeting, err := ioutil.ReadAll(res.Body) res.Body.Close() if err != nil { log.Fatal(err) } fmt.Printf("%s", greeting) } 14

httptest.ResponseRecorder httptest.ResponseRecorder is an implementation of http.ResponseWriter that records its mutations for later inspection in tests. type ResponseRecorder struct { Code int // the HTTP response code from WriteHeader HeaderMap http.Header // the HTTP response headers Body *bytes.Buffer // if non-nil, the bytes.Buffer to append written data to Flushed bool } 15

httptest.ResponseRecorder in action By passing a ResponseRecorder into an HTTP handler we can inspect the generated response. // +build ignore,OMIT package main import ( "fmt" "log" "net/http" "net/http/httptest" ) func main() { handler := func(w http.ResponseWriter, r *http.Request) { http.Error(w, "something failed", http.StatusInternalServerError) } req, err := http.NewRequest("GET", "http://example.com/foo", nil) if err != nil { log.Fatal(err) } w := httptest.NewRecorder() handler(w, req) fmt.Printf("%d - %s", w.Code, w.Body.String()) } 16

Race Detection A data race occurs when two goroutines access the same variable concurrently and at least one of the accesses is a write. To help diagnose such bugs, Go includes a built-in data race detector. Pass the -race flag to the go tool to enable the race detector: $ go test -race mypkg // to test the package $ go run -race mysrc.go // to run the source file $ go build -race mycmd // to build the command $ go install -race mypkg // to install the package 17

Testing with concurrency When testing concurrent code, there's a temptation to use sleep;

it's easy and works most of the time. But "most of the time" isn't always and flaky tests result. We can use Go's concurrency primitives to make flaky sleep-driven tests reliable. 18

Finding errors with static analysis: vet The vet tool checks code for common programmer mistakes: bad printf formats,

bad build tags,

bad range loop variable use in closures,

useless assignments,

unreachable code,

bad use of mutexes,

and more. Usage: go vet [package] 19

Testing from the inside Most tests are compiled as part of the package under test. This means they can access unexported details, as we have already seen. 20

Testing from the outside Occasionally you want to run your tests from outside the package under test. (Test files as package foo_test instead of package foo .) This can break dependency cycles. For example: The testing package uses fmt .

package uses . The fmt tests must import testing .

tests must import . So the fmt tests are in package fmt_test and can import both testing and fmt . 21

Mocks and fakes Go eschews mocks and fakes in favor of writing code that takes broad interfaces. For example, if you're writing a file format parser, don't write a function like this: func Parse(f *os.File) error instead, write functions that take the interface you need: func Parse(r io.Reader) error (An *os.File implements io.Reader , as does bytes.Buffer or strings.Reader .) 22

Subprocess tests Sometimes you need to test the behavior of a process, not just a function. func Crasher() { fmt.Println("Going down in flames!") os.Exit(1) } To test this code, we invoke the test binary itself as a subprocess: func TestCrasher(t *testing.T) { if os.Getenv("BE_CRASHER") == "1" { Crasher() return } cmd := exec.Command(os.Args[0], "-test.run=TestCrasher") cmd.Env = append(os.Environ(), "BE_CRASHER=1") err := cmd.Run() if e, ok := err.(*exec.ExitError); ok && !e.Success() { return } t.Fatalf("process ran with err %v, want exit status 1", err) } 23