Breakdown of request time including time spent within RPC’s.

Google Cloud Platform recently released Stackdriver Trace which helps highlight and debug performance bottlenecks within our software. The library gives us the ability to inspect detailed latency information for each step of a programs execution.

Out of the box Google Cloud Go library includes some really nice plumbing for passing trace context between HTTP requests. However, this is currently not the case when tracing remote procedure calls with gRPC.

Docs from Google Trace library

To associate our remote child processes with their parent spans we’ll need to pass a special trace context header along with the gRPC request.

TRACE_ID/SPAN_ID;o=TRACE_TRUE"

Similar to the Google Cloud trace dial option we can create a new client interceptor that adds the google cloud context header to the gRPC metadata before invoking the request.

When the request lands on the downstream service we can use a server interceptor to parse the trace header from the request metadata and create a new child span of the parent trace.

This gives us the ability to create new child spans within the downstream service that are associated with the parent span from the RPC.