The State of Go Where we are in February 2017 Francesc Campoy Google Developer Advocate

Time flies Go 1.6 is one year old (Happy Birthday!) Go 1.7 is already 6 months old! Go 1.8 was released on February 16th. 2

Notes The slides are available on talks.golang.org/2017/state-of-go.slide Most of the code examples won't run except locally and using Go 1.8. The playground still runs Go 1.7. 3

Agenda Changes since Go 1.7: The Language

The Standard Library

The Runtime

The Tooling

The Community 4

Changes to the language 5

Conversion rules How many times have you found yourself with two types that were almost equal? Let's say you define Person : type Person struct { Name string AgeYears int SSN int } And that for some reason, like JSON you also have: var aux struct { Name string `json:"full_name"` AgeYears int `json:"age"` SSN int `json:"social_security"` } 6

Conversion rules In order to convert aux to type Person you needed to do: type Person struct { Name string AgeYears int SSN int } return Person{ Name: aux.Name, AgeYears: aux.AgeYears, SSN: aux.SSN } 7

Conversion rules Since Go 1.8 you can simply do: return Person(aux) Both types still need to have: same sequence of fields (the order matters)

of fields (the order matters) corresponding fields with same type. 8

Conversion rules A non-constant value x can be converted to type T in any of these cases: x is assignable to T.

x's type and T have identical underlying types.

x's type and T are unnamed pointer types and their pointer base types have identical underlying types.

... 9

Conversion rules A non-constant value x can be converted to type T in any of these cases: x is assignable to T.

ignoring struct tags , x's type and T have identical underlying types.

, x's type and T have identical underlying types. ignoring struct tags , x's type and T are unnamed pointer types and their pointer base types have identical underlying types.

, x's type and T are unnamed pointer types and their pointer base types have identical underlying types. ... 10

Ports to other platforms 11

Ports to other platforms 32-bit MIPS big-endian ( linux/mips )

) little-endian ( linux/mipsle ) - requires Floating Point Unit Go on DragonFly BSD now requires DragonFly 4.4.4+. Go on OpenBSD now requires OpenBSD 5.9+. Plan 9 is now better! 12

Ports to other platforms Go 1.8 supports OS X 10.8+. Likely last time we support 10.8. ARM: Go 1.8 is the last version to support ARMv5E and ARMv6 processors. Go 1.9 will require ARMv6K. Will it work on my platform? go tool dist -check-armv6k 13

Tools 14

Fix Fixes the import path "golang.org/x/net/context" to "context" . package main import "golang.org/x/net/context" func main() { ctx := context.Background() doSomething(ctx) } func doSomething(ctx context.Context) { // doing something } Simply run the command below: #!/bin/bash go tool fix -diff -force=context state-of-go/tools/gofix.go Drop the -diff flag to rewrite the files. 15

Vet "Vet is stricter in some ways and looser where it previously caused false positives." Example of extra check: // +build ignore,OMIT package main import ( "io" "log" "net/http" "os" ) func main() { res, err := http.Get("https://golang.org") defer res.Body.Close() if err != nil { log.Fatal(err) } io.Copy(os.Stdout, res.Body) } govet detects the problem statically: #!/bin/bash go vet state-of-go/tools/govet.go 16

SSA everywhere! The SSA backend: generates more compact and efficient code

is a better platform for optimizations For 32-bit ARM systems this means 20-30% speed up! For others (where SSA was used already) gains are 0-10%. 17

SSA everywhere 18

Default GOPATH Yay! When GOPATH is not defined, the tool will use: $HOME/go on Unix

on Unix %USERPROFILE%\go on Windows 19

go bug Easier way to create bugs including all relevant information. Example: #! /bin/bash go bug 20

Runtime 21

Detection of concurrent map accesses Improvement on Go 1.6. // +build ignore,OMIT package main import ( "fmt" "sync" ) func main() { const workers = 100 // what if we have 1, 2, 25? var wg sync.WaitGroup wg.Add(workers) m := map[int]int{} for i := 1; i <= workers; i++ { go func(i int) { for j := 0; j < i; j++ { m[i]++ } wg.Done() }(i) } wg.Wait() fmt.Println(m) } Outputs: fatal error: concurrent map read and map write fatal error: concurrent map writes 22

Mutex Contention Profiling Profile your benchmarks and the contention on your mutexes. go test bench=. -mutexprofile=mutex.out Alternatively, activate contention profiling with this new method. runtime.SetMutexProfileFraction Note: For now sync.RWMutex is not profiled. 23

Mutex Contention Profiling Let's write a program to count how many times each factor appears from 2 to N. Example N = 10: Factorizations: 2: 2 3: 3 4: 2 2 5: 5 6: 2 3 7: 7 8: 2 2 2 9: 3 3 10: 2 5 Count: 2: 8 3: 4 5: 2 7: 1 24

Mutex Contention Profiling Which option is better? Wide protected region: // +build ignore,OMIT package main import ( "flag" "fmt" "sort" "sync" ) func main() { n := flag.Int("n", 10, "maximum number to consider") flag.Parse() type pair struct{ n, c int } var pairs []pair for n, c := range countFactorsWideSection(*n) { pairs = append(pairs, pair{n, c}) } sort.Slice(pairs, func(i, j int) bool { return pairs[i].n < pairs[j].n }) for _, p := range pairs { fmt.Printf("%3d: %3d

", p.n, p.c) } } func countFactorsNarrowSection(n int) map[int]int { m := map[int]int{} var mu sync.Mutex var wg sync.WaitGroup wg.Add(n - 1) for i := 2; i <= n; i++ { go func(i int) { // NARROW OMIT for _, f := range factors(i) { mu.Lock() // HL m[f]++ mu.Unlock() // HL } wg.Done() }(i) } wg.Wait() return m } func countFactorsWideSection(n int) map[int]int { m := map[int]int{} var mu sync.Mutex var wg sync.WaitGroup wg.Add(n - 1) for i := 2; i <= n; i++ { go func(i int) { mu.Lock() for _, f := range factors(i) { m[f]++ } mu.Unlock() wg.Done() }(i) } wg.Wait() return m } func countFactorsSeq(n int) map[int]int { m := map[int]int{} for i := 2; i <= n; i++ { for _, f := range factors(i) { // HL m[f]++ // HL } // HL } return m } func factors(v int) []int { var fs []int for v > 1 { for f := 2; f <= v; f++ { if v%f == 0 { v = v / f fs = append(fs, f) break } } } return fs } Narrow protected region: // +build ignore,OMIT package main import ( "flag" "fmt" "sort" "sync" ) func main() { n := flag.Int("n", 10, "maximum number to consider") flag.Parse() type pair struct{ n, c int } var pairs []pair for n, c := range countFactorsWideSection(*n) { pairs = append(pairs, pair{n, c}) } sort.Slice(pairs, func(i, j int) bool { return pairs[i].n < pairs[j].n }) for _, p := range pairs { fmt.Printf("%3d: %3d

", p.n, p.c) } } func countFactorsNarrowSection(n int) map[int]int { m := map[int]int{} var mu sync.Mutex var wg sync.WaitGroup wg.Add(n - 1) for i := 2; i <= n; i++ { go func(i int) { for _, f := range factors(i) { mu.Lock() m[f]++ mu.Unlock() } wg.Done() }(i) } wg.Wait() return m } func countFactorsWideSection(n int) map[int]int { m := map[int]int{} var mu sync.Mutex var wg sync.WaitGroup wg.Add(n - 1) for i := 2; i <= n; i++ { go func(i int) { // WIDE OMIT mu.Lock() // HL for _, f := range factors(i) { m[f]++ } mu.Unlock() // HL wg.Done() }(i) } wg.Wait() return m } func countFactorsSeq(n int) map[int]int { m := map[int]int{} for i := 2; i <= n; i++ { for _, f := range factors(i) { // HL m[f]++ // HL } // HL } return m } func factors(v int) []int { var fs []int for v > 1 { for f := 2; f <= v; f++ { if v%f == 0 { v = v / f fs = append(fs, f) break } } } return fs } 25

Benchmark $ go test -bench=. 26

Benchmarking with Mutex Contention $ go test -bench=. -mutexprofile=mutex.out 27

Analyzing the Mutex Contention Profile $ go tool pprof runtime.test mutex.out Entering interactive mode (type "help" for commands) (pprof) list 0 5.38s (flat, cum) 43.97% of Total . . 34: mu.Lock() . . 35: m[f]++ . 5.38s 36: mu.Unlock() 0 6.86s (flat, cum) 56.03% of Total . . 53: mu.Lock() . . 54: for _, f := range factors(i) { . . 55: m[f]++ . . 56: } . 6.86s 57: mu.Unlock() 28

So much contention ... 29

Contention by CPU 30

Comparing it to sequential algorithm 31

Comparing it to sequential algorithm (zoom) 32

Performance 33

GC history in tweets 34

go 1.5 35

go 1.6 36

go 1.7 37

go 1.8 (beta 1) 38

go 1.8 (beta 1) CPU 39

defer is faster name old time/op new time/op delta Defer-4 101ns ± 1% 66ns ± 0% -34.73% (p=0.000 n=20+20) Defer10-4 93.2ns ± 1% 62.5ns ± 8% -33.02% (p=0.000 n=20+20) DeferMany-4 148ns ± 3% 131ns ± 3% -11.42% (p=0.000 n=19+19) 40

cgo is also faster! name old time/op new time/op delta CgoNoop-8 93.5ns ± 0% 51.1ns ± 1% -45.34% (p=0.016 n=4+5) Source: dave.cheney.net 41

Changes to the standard library 42

Sorting Exercise: Given a slice of Person var p []Person Print the slice sorted by name, age, and SSN. sort.Sort(byName(p)) sort.Sort(byAge(p)) sort.Sort(bySSN(p)) Easy, right? 43

Sorting Well, you forgot about this part. type byName []Person func (b byName) Len() int { return len(b) } func (b byName) Less(i, j int) bool { return b[i].Name < b[j].Name } func (b byName) Swap(i, j int) { b[i], b[j] = b[j], b[i] } type byAge []Person func (b byAge) Len() int { return len(b) } func (b byAge) Less(i, j int) bool { return b[i].AgeYears < b[j].AgeYears } func (b byAge) Swap(i, j int) { b[i], b[j] = b[j], b[i] } type bySSN []Person func (b bySSN) Len() int { return len(b) } func (b bySSN) Less(i, j int) bool { return b[i].SSN < b[j].SSN } func (b bySSN) Swap(i, j int) { b[i], b[j] = b[j], b[i] } 44

sort.Slice Since Go 1.8 you can simply write this: sort.Slice(p, func(i, j int) bool { return p[i].Name < p[j].Name }) sort.Slice(p, func(i, j int) bool { return p[i].AgeYears < p[j].AgeYears }) sort.Slice(p, func(i, j int) bool { return p[i].SSN < p[j].SSN }) Also new SliceStable and SliceIsSorted . 45

Benchmark N=1 go test -bench=. BenchmarkSortSort-8 10000000 145 ns/op BenchmarkSortSlice-8 10000000 190 ns/op N=10 go test -bench=. BenchmarkSortSort-8 2000000 918 ns/op BenchmarkSortSlice-8 1000000 1776 ns/op N=100 go test -bench=. BenchmarkSortSort-8 100000 16588 ns/op BenchmarkSortSlice-8 50000 39035 ns/op N=1000 go test -bench=. BenchmarkSortSort-8 5000 320951 ns/op BenchmarkSortSlice-8 3000 446677 ns/op N=10000 go test -bench=. BenchmarkSortSort-8 500 3644480 ns/op BenchmarkSortSlice-8 300 4962263 ns/op N=100000 go test -bench=. BenchmarkSortSort-8 30 43573572 ns/op BenchmarkSortSlice-8 20 60861706 ns/op Benchmark ran on my MacBook Pro (8 cores), simply indicative. 46

Benchmark 47

Benchmark (log/log) 48

Plugins Define a plugin: package main import "fmt" var V int func F() { fmt.Printf("Hello, number %d

", V) } Then build it: go build -buildmode=plugin Note: This currently works only on Linux. 49

Plugins p, err := plugin.Open("plugin_name.so") if err != nil { panic(err) } v, err := p.Lookup("V") if err != nil { panic(err) } f, err := p.Lookup("F") if err != nil { panic(err) } *v.(*int) = 7 f.(func())() // prints "Hello, number 7" 50

Plugins demo Demo video: twitter.com/francesc Source code: github.com/campoy/golang-plugins 51

HTTP shutdown Added Shutdown method to http.Server . Example: Call Shutdown when a signal is received: // subscribe to SIGINT signals quit := make(chan os.Signal) signal.Notify(quit, os.Interrupt) srv := &http.Server{Addr: ":8080", Handler: http.DefaultServeMux} go func() { <-quit log.Println("Shutting down server...") if err := srv.Shutdown(context.Background()); err != nil { log.Fatalf("could not shutdown: %v", err) } }() 52

HTTP shutdown Check why the server stopped. http.HandleFunc("/", handler) err := srv.ListenAndServe() if err != http.ErrServerClosed { log.Fatalf("listen: %s

", err) } log.Println("Server gracefully stopped") 53

HTTP/2 http.Response now satisfies the http.Pusher interface. type Pusher interface { Push(target string, opts *PushOptions) error } A simple example: func rootHandler(w http.ResponseWriter, r *http.Request) { if p, ok := w.(http.Pusher); ok { err := p.Push("/style.css", nil) if err != nil { log.Printf("could not push: %v", err) } } fmt.Fprintln(w, html) } 54

HTTP/2 // +build ignore,OMIT package main import ( "fmt" "go/build" "log" "net/http" "path/filepath" ) var cert, key string func init() { pkg, err := build.Import("golang.org/x/talks/content/2017/state-of-go/stdlib/http2", ".", build.FindOnly) if err != nil { log.Fatal(err) } cert = filepath.Join(pkg.Dir, "cert.pem") key = filepath.Join(pkg.Dir, "key.pem") } func main() { http.HandleFunc("/", rootHandler) http.HandleFunc("/style.css", cssHandler) go func() { log.Fatal(http.ListenAndServeTLS("127.0.0.1:8081", cert, key, nil)) }() log.Fatal(http.ListenAndServe("127.0.0.1:8080", nil)) } func rootHandler(w http.ResponseWriter, r *http.Request) { if p, ok := w.(http.Pusher); ok { // HL err := p.Push("/style.css", nil) // HL if err != nil { log.Printf("could not push: %v", err) } } fmt.Fprintln(w, html) } func cssHandler(w http.ResponseWriter, r *http.Request) { fmt.Fprintln(w, css) } const ( html = ` <html> <head> <link rel="stylesheet" href="/style.css"> <title>HTTP2 push test</title> </head> <body> <h1>Hello</h1> </body> </html> ` css = ` h1 { color: red; text-align: center; text-shadow: green 0 0 40px; font-size: 10em; } ` ) HTTP: localhost:8080

HTTP/2: localhost:8081 55

HTTP/2 HTTP HTTP/2 56

More context support Since Go 1.7: net

net/http

os/exec Since Go 1.8: http.Server.Shutdown

database/sql

net.Resolver 57

A couple more changes too Go 1.8 release notes 58

The community 59

Women Who Go 16 chapters already! www.womenwhogo.org 60

Go meetups Gophers all around the world! go-meetups.appspot.com 61

Go 1.8 release party, February 16th Go 1.8 ships soon! Go meetups are organising to hold a release party on the 16th of February. Join the party!!! 63