End-to-End Encrypted RPC/PubSub over Tor

Using Crossbar/Autobahn with Tor and Encryption

The Tor Project provides an overlay network that hides the network location of TCP streams. Both clients and servers are supported. Crossbar.io provides routed Remote Procedure Calls (RPC) and Publish/Subscribe (PubSub) messaging (called Web Application Messaging Protocol or WAMP) via many transports, including WebSockets. The project has recently added native support for Tor (via txtorcon).

This is a two-part blog post, with the high-level overview (from a Tor perspective) found at the Tor blog as Secure Messaging with Onion Services. The following post describes the detailed technical aspects of the setup.

Overview The txtorcon documentation (clearnet) should be re-built whenever GitHub changes happen. GitHub provides a WebHook service which can notify you whenever interesting changes happen. These notifications take the form of an HTTP POST. Ordinarily, accepting such an HTTP request and re-building the documentation would be straightforward. However, an Onion service is trying to hide its network location and giving GitHub some way to call this directly would break that. So, we will run an agent on a public IP that listens for GitHub WebHook requests and forwards them as “publish” events over Tor to the Crossbar.io router. There will be another agent connected to the router that will subscribe to these forwarded GitHub events. This agent re-builds the documentation that is served by the txtorcon Onion service. There are three parts in the above diagram that we will deal with here: configure the Crossbar.io router to work over Tor

write the GitHub Agent (in Python)

write the Doc Builder (in Python) All of the code for this is on GitHub as txtorcon-documentation-builder with some excerpts from it appearing in this post. In this post and the included code above, we set up a location-hidden Crossbar.io instance listening on a Tor onion service with a publicly-facing component that verifies and forwards GitHub WebHook requests. These requests are acted on by a second “builder” component that re-builds the txtorcon documentation. This keeps the network-location of the documentation host and builder unknown. In a bonus section, we detail how to also end-to-end encrypt the messages exchanged between the WebHook listener and the builder components.

The Crossbar Router Crossbar.io is a Twisted application that provides a robust implementation of a “WAMP router”, routing RPC and PubSub messages between various connected clients. It supports many transports (stdin/out, Unix sockets, TCP sockets and WebSockets). For our purposes we will use WebSockets over Tor, which Crossbar natively supports. This is configured using the onion transport type, which requires a “control” connection to the Tor daemon. This is so we can use the ADD_ONION command to install a new Onion service. This will allocate a random local-only TCP port to which the resulting traffic from Tor will be sent. Crossbar also supports encrypting the actual RPC or PubSub payloads. Although the Tor traffic is already end-to-end encrypted, this only gets data from one client to the router. The “cryptobox” feature of Crossbar allows us to encrypt the message payloads for other clients. This uses NaCl “Box” operations. After securely exchanging two clients’ public keys, these two clients of a Crossbar router can exchange RPC and PubSub messages without the Crossbar process being able to see any of the payload. Of course, the router can still analyze any metadata, such as: how often the RPC is called; the size of the payload; between which clients the communication is happening; etcetera. Nonetheless, being able to hide the actual payload is a nice feature. To authenticate the clients to the router, we will use Crossbar’s “WAMP Cryptosign” authentication, which uses the NaCl “crypto_sign” algorithm to make a Curve 25519 signature of a challenge from Crossbar. Setting Up Crossbar After installing Crossbar via one of the methods in Crossbar’s installation guide we need to provide some relevant configuration. First of all is our Onion service, with a stanza similar to the following: "workers" : [ { "type" : "router" , "transports" : [ { "type" : "websocket" , "endpoint" : { "type" : "onion" , "port" : 8080 , "private_key_file" : "hskey" , "tor_control_endpoint" : { "type" : "unix" , "path" : "/var/run/tor/control" } } } ] } ] The above will result in a .crossbar/hskey file being created with the private-key of the new service; on a subsequent start-up of this same configuration the same Onion service will again be launched. You can also provide an absolute path to this file if you prefer (relative paths are relative to the crossbar directory). You will have to look in Crossbar’s log file for the name of the created service (or for any errors from Tor). It will be something like m6dazoly4sqnoqrm.onion . Note that anyone with the private key can create a service listening on this onion address, so you must keep the key secret. Now we can tell our two client services (the GitHub WebHook endpoint and the actual documentation builder) to connect with the WebSocket URI ws://m6dazoly4sqnoqrm.onion:8080/ . If the crossbar service is running on the same machine as the documentation-building service, we could configure a local Unix socket for it to connect on; for brevity I’m leaving that out of this post. If all went well, running crossbar start should result in something like this: 2017-10-01T00:00:25-0600 [Router 26361] Realm 'agent' started 2017-10-01T00:00:25-0600 [Controller 26355] Router "worker-001": realm 'realm-001' (named 'agent') started 2017-10-01T00:00:25-0600 [Router 26361] role role-001 on realm realm-001 started 2017-10-01T00:00:25-0600 [Controller 26355] Router "worker-001": role 'role-001' (named 'github') started on realm 'realm-001' 2017-10-01T00:00:25-0600 [Router 26361] role role-002 on realm realm-001 started 2017-10-01T00:00:25-0600 [Controller 26355] Router "worker-001": role 'role-002' (named 'builder') started on realm 'realm-001' 2017-10-01T00:00:25-0600 [Router 26361] WampWebSocketServerFactory starting on '[redacted]' 2017-10-01T00:00:25-0600 [Controller 26355] Router "worker-001": transport 'transport-001' started 2017-10-01T00:00:25-0600 [Router 26361] WampWebSocketServerFactory starting on 60220 2017-10-01T00:00:25-0600 [Router 26361] Uploading descriptors can take more than 30s 2017-10-01T00:00:25-0600 [Router 26361] Created hidden-service at [redacted] 2017-10-01T00:00:25-0600 [Router 26361] Created '[redacted]', waiting for descriptor uploads. 2017-10-01T00:00:59-0600 [Router 26361] Uploaded '[redacted]' to '$[redacted]' 2017-10-01T00:00:59-0600 [Router 26361] Listening on Tor onion service [redacted] with local port 60220 2017-10-01T00:00:59-0600 [Controller 26355] Router "worker-001": transport 'transport-002' started 2017-10-01T00:00:59-0600 [Controller 26355] Local node configuration applied successfully! 2017-10-01T00:05:01-0600 [Router 26361] session "2099533749194636" joined realm "agent" 2017-10-01T00:05:33-0600 [Router 26361] session "5229690000667752" joined realm "agent" Setting Up a Crossbar Client On a machine with a public IP address, we will run the GitHub WebHook client. This will do two things: listen on 443 for HTTPS connections from GitHub;

and connect to Crossbar via Tor turning any WebHook calls into “publish” events The documentation builder will subscribe to these topics and trigger builds when the source code changes. (Note again that Crossbar has native support for this, but we don’t want Crossbar to be on a public IP address for this exercise). We will use Klein as our Web server and txacme to get a (free!) Let’s Encrypt certificate for our server. Using the “Component” API in Crossbar, configuration of our client will look something like this (in Python, using Twisted): from autobahn.twisted.component import Component , run from twisted.internet.endpoints import clientFromString hook = Component ( transports = [ { "endpoint" : clientFromString ( reactor , u'tor:m6dazoly4sqnoqrm.onion:5000' ), "url" : u"ws://m6dazoly4sqnoqrm.onion:5000/" , } ], realm = u"agent" , authentication = { u"cryptosign" : { u"authid" : u"agent" , u"authrole" : u"github" , u"privkey" : u"[redacted]" , } } ) if __name__ == '__main__' : run ([ hook ]) One thing to note is that the “endpoint” configuration is an actual Twisted IStreamClientEndpoint instance, and the tor: prefix is added by txtorcon.

Receiving the WebHooks This agent also needs to listen for the GitHub POST requests, for which we will use Klein, a modern Twisted Web development framework (sometimes likened to Flask). A WAMP session goes through a predictable lifecycle of four possible events: connect , join , leave , disconnect (in that order). The connect and disconnect are self-explantory transport-level events. The join event fires after we’ve connected to a router and succcessfully authenticated. You may add a callback for any event by calling component_instance.on('event_name', func) or there are convenience decorators, which we will use: from twisted.web.server import Site import klein # from above, 'hook' is a autobahn.twisted.component.Component instance @hook.on_join @defer.inlineCallbacks def join ( session , details ): app = klein . app . Klein () @app.route ( '/webhook/github' , methods = [ 'POST' ]) def github_webhook ( request ): pass # details removed; see repository site = Site ( app . resource ()) ep = serverFromString ( reactor , 'le:/tmp/certs:tcp:443' ) yield ep . listen ( site ) This gives us a site listening on port 443 using Let’s Encrypt certificates thanks to txacme (which will also renew them). In this instance, they’ll be stored in /tmp/certs . The “see details” code includes checking the signatures from GitHub, so we can be assured these are legitimate requests.

Setting Up Github Once the above agent is in place on a public machine (and listening on 443) we can configure GitHub to send us POSTs. Very briefly (as there are many good resources to help with this part), we ask for TLS-verified POST requests to our public machine on the URI we’re listening on. Find this configuration in the “settings” tab of your repository: Point it at the machine where the agent is running. This is a machine with a public IP address.

The Documentation Builder So now we have: a Crossbar.io router on a private machine and a WebHook agent on a public machine. The Crossbar.io machine is connected only via Tor, with no incoming connections allowed (all Tor connections are outbound). (Updated: very briefly, your tor client makes an outbound circuit to one of the “rendezvous points” to which the service-providing tor has previously made its own outbound circuits; the rendezvous point glues them together. There will be a listen() call happening on localhost:random_port or a Unix socket but no incoming TCP connections from the Internet.) The next step is to add the agent that actually re-builds the documentation on the machine serving the txtorcon documentation. This machine also is connected to the outside world only via Tor. This agent will connect to Crossbar.io via Tor and listen for “publish” events – that is, the GitHub WebHook announcements that are now being turned into WAMP “publish” messages. For the full details, see the repository. The important parts are how we connect to the Tor onion service. builder = Component ( transports = [ { "endpoint" : clientFromString ( reactor , u'tor:m6dazoly4sqnoqrm.onion:5000' ), "url" : u"ws://m6dazoly4sqnoqrm.onion:5000/" , } ], realm = u"builder" , ) @builder.subscribe ( u'webhook.github.push' ) @defer.inlineCallbacks def _github_push ( ** kw ): if kw [ 'ref' ] != 'refs/heads/master' : return # set PATH to our builder's venv # '_run' is a helper that spawns a process # 'disthome' points to our checkout yield _run ( disthome , '/usr/bin/git' , 'pull' ) yield _run ( join ( disthome , 'docs' ), '/usr/bin/make' , 'html' )

Conclusion We have set up a Crossbar.io listening on a Tor Project onion service with a publicly-facing component that verifies and forwards GitHub WebHook requests. These requests are acted on by a second “builder” component that re-builds the txtorcon documentation. This keeps the network-location of the documentation host unknown.