We build a lot of microservices in golang, which primarily use HTTP/JSON APIs to communicate with each other. Over the years, we’ve used a number of golang HTTP client libraries to compose and execute HTTP requests, mainly to cut down on the the boilerplate required to use go’s http library.

I eventually had enough itches to scratch that I built a new library: github.com/gemalto/requester

It makes it simple to create and execute HTTP requests, and handle responses. It attempts to address issues we’ve had with other libraries:

Full access to http package’s primitives if needed Simple, idiomatic go Support for context.Context Embeddable: uses the function option pattern, rather than fluent API pattern Test utils for writing HTTP unit tests

Here’s a sample of building, sending, and receiving an HTTP request using the native http package:

bodyBytes, err := json.Marshal(reqBodyStruct)

if err != nil { return err }



bodyBytes, err := json.Marshal(requestBody)

if err != nil {

panic(err)

}



req, err := http.NewRequest("POST", "http://api.com/resources/", bytes.NewReader(bodyBytes))

if err != nil {

panic(err)

}

req.Header.Set("Content-Type", "application/json")

req.Header.Set("Accept", "application/json")



resp, err := http.DefaultClient.Do(req)

if err != nil {

panic(err)

}



if resp.StatusCode != 201 {

panic(errors.New("expected code 201"))

}



respBody, _ := ioutil.ReadAll(resp.Body)

var r Resource

if err := json.Unmarshal(respBody, &r); err != nil {

panic(err)

}



fmt.Printf("%d %s %v", resp.StatusCode, string(respBody), r)

requester uses functional options to configure the request, and folds request building, execution, and response handling into a single call:

var r Resource



resp, body, err := requester.ReceiveContext(ctx, &r,

requester.JSON(false),

requester.Body(requestBody),

requester.Post("http://api.com/resources/"),

requester.ExpectCode(201),

)

if err != nil {

panic(err)

}



fmt.Printf("%d %s %v", resp.StatusCode, string(body), r)

Here’s a more complicated example demonstrating many features:

func Example_everything() {



type Resource struct {

ID string `json:"id"`

Color string `json:"color"`

}



s := httptest.NewServer(requester.MockHandler(201,

requester.JSON(true),

requester.Body(&Resource{Color: "red", ID: "123"}),

))

defer s.Close()



r := httptestutil.Requester(s,

requester.Post("/resources?size=big"),

requester.BearerAuth("atoken"),

requester.JSON(true),

requester.Body(&Resource{Color: "red"}),

requester.ExpectCode(201),

requester.Header("X-Request-Id", "5"),

requester.QueryParam("flavor", "vanilla"),

requester.QueryParams(&struct{ Type string `url:"type"` }{Type: "upload"}),

requester.Client(

httpclient.SkipVerify(true),

httpclient.Timeout(5*time.Second),

httpclient.MaxRedirects(3),

),

)



r.MustApply(requester.DumpToStderr())

httptestutil.Dump(s, os.Stderr)



serverInspector := httptestutil.Inspect(s)

clientInspector := requester.Inspect(r)



var resource Resource



resp, body, err := r.Receive(&resource)

if err != nil {

panic(err)

}







fmt.Println("client-side request url path:", clientInspector.Request.URL.Path)

fmt.Println("client-side request query:", clientInspector.Request.URL.RawQuery)

fmt.Println("client-side request body:", clientInspector.RequestBody.String())



ex := serverInspector.LastExchange()

fmt.Println("server-side request authorization header:", ex.Request.Header.Get("Authorization"))

fmt.Println("server-side request request body:", ex.RequestBody.String())

fmt.Println("server-side request response body:", ex.ResponseBody.String())



fmt.Println("client-side response body:", clientInspector.ResponseBody.String())



fmt.Println("response status code:", resp.StatusCode)

fmt.Println("raw response body:", string(body))

fmt.Println("unmarshaled response body:", resource)



// Output:

// client-side request url path: /resources

// client-side request query: size=big&flavor=vanilla&type=upload

// client-side request body: {

// "id": "",

// "color": "red"

// }

// server-side request authorization header: Bearer atoken

// server-side request request body: {

// "id": "",

// "color": "red"

// }

// server-side request response body: {

// "id": "123",

// "color": "red"

// }

// client-side response body: {

// "id": "123",

// "color": "red"

// }

// response status code: 201

// raw response body: {

// "id": "123",

// "color": "red"

// }

// unmarshaled response body: {123 red}



}

The core of the API is:

// just build a request

Request(...Option) (*http.Request, error)

RequestContext(context.Context, ...Option) (*http.Request, error) // build a request and execute it

Send(...Option) (*http.Response, error)

SendContext(context.Context, ...Option) (*http.Response, error) // build and send a request, and handle the response

Receive(interface{}, ...Option) (*http.Response, []byte, error)

ReceiveContext(context.Context, interface{}, ...Option) (*http.Response, []byte, error)

Receive/ReceiveContext reads and closes the response body, returns it as a byte slice, and also attempts to unmarshal it into a target value, if one is provided.

These functions are defined on the package, as well as on *requester.Requester . The package functions just delegate to a requester.DefaultRequester .

Each of these accept a varargs of functional options. Option is defined as:

type Option interface {

Apply(*Requester) error

}



type OptionFunc func(*Requester) error

Options are simply functions which modify the attributes of a *Requester . The attributes of Requester control how requests are created, executed, and how responses are handled:

type Requester struct {

Method string

URL *url.URL

Header http.Header

GetBody func() (io.ReadCloser, error)

ContentLength int64

TransferEncoding []string

Close bool

Host string

Trailer http.Header

QueryParams url.Values

Body interface{}

Marshaler Marshaler

Unmarshaler Unmarshaler

Doer Doer

Middleware []Middleware

}

Requester uses an interface called Doer to execute requests. *http.Client implements Doer . The Doer interface also enables support for client-side middleware.

Requester also includes tools for writing unit tests for both client and server code:

requester.Inspector captures the client side of a request/response exchange

captures the client side of a request/response exchange requester.Dump(io.Writer) is an option which dumps the full request and response to an output. Other options dump to os.Stdout , os.Stderr , and the test log

is an option which dumps the full request and response to an output. Other options dump to , , and the test log httptestutil.Requester(*httptest.Server) returns a *Requester preconfigured to talk to an *httptest.Server .

returns a preconfigured to talk to an . httptestutil.Dump(*httptest.Server, io.writer) installs server-side middleware which dumps requests and responses, including bodies, to a writer. Other functions dump to os.Stdout or the test log.

installs server-side middleware which dumps requests and responses, including bodies, to a writer. Other functions dump to or the test log. httptestutil.Inspect(*httptest.Server) installs server-side middleware which captures requests and responses

API changes are possible before version 1.x.x, but unexpected. The library is already being used by several in-house projects.