Practical advice for Go library authors Writing better libraries 14 July 2016 Jack Lindamood Software Engineer, Twitch

Goal of this talk Many ways to do common things Which way is preferred and why Common pitfalls of Go libraries What I wish I knew 2

Naming things 3

General naming advice Generally imports are not renamed Standard practice is PACKAGE.FUNCTION() Think of package as part of the name // In Java, the package name is silent Java: new BufferedReader() // In Go, the package name is where it is used Go: bufio.NewReader() 4

Naming examples Tips for structs stuttering struct names ok, but avoid if you can // bad. Too generic var c client.Client // stuttering, but accepted var ctx context.Context // optimal example var h http.Client Tips for functions package functions should never stutter // Ok. Reads as a background context context.Background() // Not ok. Context redundant context.NewContext() 5

Object creation 6

Object construction No rigid constructor like other languages Zero value sometimes works Constructor function (NewXYZ) has ramifications Using the struct or constructor function // Using the struct x := http.Client{} // Constructor function y := bytes.NewBuffer(...) 7

Zero value considerations // Examples of a zero value var c http.Client y := bytes.Buffer{} Read operations work well on nil map/slice/chan Great for making your zero value behave Less useful if struct needs background goroutines non zero/empty defaults are the best Zero struct support is viral!!!! So is nil support 8

Working with zero values Example for defaults // net/http/server.go func (srv *Server) ListenAndServe() error { addr := srv.Addr if addr == "" { addr = ":http" } // ... } Example for nil reads // bytes/buffer.go type Buffer struct { buf []byte off int } func (b *Buffer) Len() int { return len(b.buf) - b.off } 9

New() constructor function Not as terse as making zero value work Constructors can do anything. Struct initialization only does one thing. Risk people use struct w/o getting it from New Some libraries hide this with a private struct, public interface // anti-pattern.go type authImpl struct { } type Auth interface { // Tends to be very large } func NewAuth() Auth { } 10

What do these do? func directUse() { x := grpc.Client{ Port: 123, Hostname: "elaine82.sjc", } // ... } func constructor() { x := grpc.NewClient(123, "elaine82.sjc") // ... } Does constructor() do the following spawn goroutines allocate extra memory panic 11

Singleton options Go stdlib makes heavy use of singletons

not a fan of this Beware hidden singletons (expvar/http/rand/etc) General way singletons are done var std = New(os.Stderr, "", LstdFlags) // Singleton functions in package. Same as struct functions func Print(v ...interface{}) { std.Print(v...) } func (l *Logger) Print(v ...interface{}) { // ... } 12

Configuration 13

Optional config w/ functions Use NewXYZ(WithThing(), WithThing()) can be clunky Has not gained adoption type Server struct { speed int } func Speed(rate int) func(*Server) { return func(s *Server) { s.speed = rate } } func NewServer(options ...func(*Server)) { } x := NewServer(Speed(10)) 14

Config struct Gaining popularity Easy to understand all available options Does zero work for default config? Sometimes not Important to document if config is usable after being passed in Easier to work with config management // Usually JSON in config management type Config struct { Speed int } func NewServer(conf *Config) { } 15

Logging 16

Logging counter-example import "log" type Server struct {} func (s *Server) handleRequest(r *Request) { if r.user == 0 { log.Println("No user set") return } // ... } Direct print to std streams There are many logging frameworks Cannot turn it off 17

Logging advice Is it needed? Logging is a callback Log to an interface Don't assume log package Respect stdout/stderr No singleton! 18

Better logging example type Log interface { Println(v ...interface{}) } type Server struct { Logger Log } func (s *Server) log(v ...interface{}) { if s.Logger != nil { s.Logger.Println(v...) } } func (s *Server) handleRequest(r *Request) { if r.user == 0 { s.log("No user set") return } } Can we do even better? Structured logging 19

Interfaces 20

interfaces vs structs Accept interfaces, return structs Some libraries only expose interfaces. Keep all structs private. Tend to be large interfaces Usually no need to include interface from outside your package/stdlib Ok to do, just not needed

Prevents import cycles

Implicit interfaces get around this

context.Context made this difficult Means an API that uses standard objects is usually best Easier to implement 21

Let's make a random package type Rand interface { ExpFloat64() float64 Float32() float32 Float64() float64 Int() int Int31() int32 Int31n(n int32) int32 Int63() int64 Int63n(n int64) int64 Intn(n int) int NormFloat64() float64 Perm(n int) []int Read(p []byte) (n int, err error) Seed(seed int64) Uint32() uint32 } There are lots of ways to generate random things What is wrong with this? 22

Avoiding large interfaces (rand.Rand) Lots of ways to get random things rand.Rand{} would be a very large interface rand.Source is an interface. Very small. The building block. Lesson: Turn large interfaces into small interfaces with wrapper logic // A Source represents a source of uniformly-distributed // pseudo-random int64 values in the range [0, 1<<63). type Source interface { Int63() int64 Seed(seed int64) } // A Rand is a source of random numbers. type Rand struct { src Source } 23

Dealing with problems 24

When to panic Never? panic() in a spawned goroutine is the worst Panic is usually ok if: Function begins with MustXYZ()

Operations on nil // MustXYZ calls XYZ, panics on error. // Note: Users still have the option to call XYZ directly func MustCompile(str string) *Regexp { regexp, error := Compile(str) if error != nil { panic(`...`) } return regexp } 25

Checking errors Check all errors on interface function calls Especially the ones you don't expect to error! If you can't return it, always do something. Log it

Increment something When are returning errors appropriate When a promise could not be kept When an answer could not be given // Counterexample HasPermission(userID) error // Better HasPermission(userID) (bool, error) 26

Enabling debuggability for your library Especially complex libraries can have state that isn't clear to the user Suggest exposing an expvar.Var{} as a function Debug logging can help Ideally expose Stat() function for atomic integers around tracking information type Stats struct { // atomic.AddInt64(&s.stats.TotalRequests, 1) TotalRequests int64 } type Server struct { DebugLog Log stats Stats } func (s *Server) Var() expvar.Var { return expvar.Func(func() interface{} { return struct{...} } } 27

Designing for testing Complex libraries could benefit users with a simple test helper httptest is an example Abstract away time/IO/os calls with interfaces Maintain control // Package stub. For trim structs var Now = time.Now // Inside struct for most control type Scheduler struct { Now func() time.Time } func (s *Scheduler) now() time.Time { if s.Now != nil { return s.Now() } return time.Now() } 28

Concurrency 29

Channels Stdlib has very few channels in the public API Push what you would use channels for up the stack Channels are rarely parameters to functions Callbacks vs channels Mixing mutexes and channels can be dangerous Honestly, channels rarely belong in a public API 30

When to spawn goroutines Some libraries use New() to spawn their goroutines. Not ideal: Stdlib uses .Serve() functions Close() should end all daemon threads Push goroutine creation up the stack // counterexample func NewServer(..) *Server { s := &Server{...} go s.Start() return s } // ideal func userLand() { // ... s := http.Server{} go s.Serve(l) // ... } 31

When to use context.Context and when not to All blocking/long operations should be cancelable! Generally an abuse to store context.Context When to use context.Value() What other languages would use thread local for So easy to abuse Try hard not to use it Singletons and context.Value() obscure your program's state machine Context.Value() Should inform not control Seriously though, try not to use context.Value() 32

If something is hard to do, make someone else do it Great advice in library design, system design, and Dilbert life The less hard things you try to do, the less likely you'll screw up whatever you're doing. Hard things (threading/Mutexes/Deadlock/channels/encryption) Push problems up the stack Logging Goroutines Locking Corollary: Try not to do things 33

Designing for efficiency Correctness still trumps efficiency Minimizing memory allocs is usually first priority Avoid creating an API that forces memory allocations Easy to optimize internals Hard to change an API func (s *Object) Encode() []byte { // .... } func (s *Object) WriteTo(w io.Writer) { // ... } 34

Using /vendor in libraries Package management is not ideal for go libraries Don't use /vendor for libraries. Will have issues if the library has singletons Try to use implicit interfaces and injection Don't expose vendored deps as part of the library's external API Their github.com/a isn't the same as your github.com/b/vendor/github.com/a Honestly, just don't use libraries in your library npm::left-pad 35

Build tags Cross OS libraries Writing libraries that are compatible with new versions of Go Example: http.Request.Cancel channel Use build tags like // +build go1.5 Integration tests Integration testing for libraries (with // +build integration) 36

Staying clean Numerous static analysis tools for go Use almost all of them Once experienced, you can lint/grep them away Build options (travis/circle/etc) Many free continuous testing integrations for open source libraries Helps ensure code works for various Go versions 37