Announcing go-msgpack

Announcing go-msgpack, a rich msgpack codec for Go. Supports encoding/decoding to msgpack binary format, and use for net/rpc communication.

https://github.com/ugorji/go-msgpack

http://gopkgdoc.appspot.com/pkg/github.com/ugorji/go-msgpack

It provides features similar to encoding packages in the standard library (ie json, xml, gob, etc).

Supports:

Standard Marshal/Unmarshal interface. Standard field renaming via tags Encoding from any value

(struct, slice, map, primitives, pointers, interface{}, etc) Decoding into a pointer to any non-nil value

(struct, slice, map, int, float32, bool, string, etc) Decoding into a nil interface{} (big) Handles time.Time transparently. Provides a Server and Client Codec so msgpack can be used as communication protocol for net/rpc.

Usage:

// v can be interface{}, int32, bool, map[string]bool, etc dec = msgpack.NewDecoder(r, nil) err = dec.Decode(&v) enc = msgpack.NewEncoder(w, nil) err = enc.Encode(v) //methods below are convenience methods over functions above. data, err = msgpack.Marshal(v, nil) err = msgpack.Unmarshal(data, &v, nil) //RPC Communication conn, err = net.Dial("tcp", "localhost:5555") rpcCodec := msgpack.NewRPCCodec(conn) client := rpc.NewClientWithCodec(rpcCodec) ...

Why?

I was initially looking at different binary serialization formats for an application I’m developing. I looked at Thrift, Protocol Buffers, BSON, Msgpack, etc.

I finally decided on msgpack:

compact supports basic types (just like JSON. Numbers, Bool, Null, Bytes/String, Map, List) from which other data structures else can be assembled raw bytes support (which can represent binary data or strings) no schema needed (just like JSON) cross-language support has pretty good mindshare

Unfortunately, the Go library on the msgpack.org site is old, does not build, and is not “smart” like the encoding/json package. Specifically, it doesn’t allow me decode into a typed object (e.g. struct, bool, etc).

I wrote go-msgpack to give the same conveniences we’ve gotten used to (spoiled by using the encoding/json package), while being really performant.

Summary of my testing:

Decodes significantly faster (120% faster) and encodes only slightly slower (20% slower) than encoding/json Uses less disk space (40% less) May not require compression (compression only gave a 10% reduction in size).

Since compression/decompression time may be significant, this may be an important win.

Hope folks use it and enjoy using it. I know I will. Please feel free to send me feedback.

Tags: golang technology