Thursday, February 19, 2015

Thrift and Go Generate

Oh serialization wars. Every new team at some point has to standardize on a serialization system, and I've yet to see this decision end with smiles all around. I am personally biased towards protocol buffers but recently find myself living with Thrift.

I use the pants build system, which is great for Python, Java and Scala. However, I also use a lot of Go, which doesn't have support from pants. Go's build system and tooling is very complete, allowing me to achieve the same ends without writing pants BUILD files and targets. For Thrift, the recent addition of go generate allows me to generate my thrift bindings from the go toolchain.

Go generate is a new tool in go1.4 that allows a comment directive to run external commands. The documentation and design doc are fairly complete (caveat: the -run flag is not yet implemented!). And it's really quite simple. Just drop a comment into any go file in your package:

 //go:generate thrift -r --gen go --out . foo.thrift  

That's it. go generate runs thrift and generates the go bindings.

Since we check in our entire workspace (including vendored dependencies, I should talk about why this is awesome later), we use thrift namespaces heavily and things start to look a bit different. My go:generate directive looks a bit different:

 //go:generate thrift -r --gen go -o $GOPATH/src/ $(git rev-list --show-toplevel )/thrift/foo.thrift  

We live with the gen-go import prefix:

 import (  
    "gen-go/foo"  
 )  

The result is the ability to generate all of our generate thrift-go code via go generate:

 $ go generate all