For those of you who might have read the previous post I did on Docker + NATS for Microservices, you’ll recall we took an in-depth look at using Docker Compose with NATS.

That blog post was done a few days prior to DockerCon, and as expected Docker made many interesting announcements at DockerCon. One of the main things announced that week was Docker 1.12.

1.12 includes Swarm mode which can be used for assembling a cluster of Docker Engines and running workloads on them.

In this blog post, we share a basic example of how showing some of these new features available in Docker 1.12 to bring up a NATS cluster that our clients running can then connect against.

Requirements

Docker Engine 1.12

Docker Swarm cluster prepared with a manager and at least a couple of workers https://docs.docker.com/engine/swarm/

Latest release of the NATS Server, which supports cluster auto-discovery for clients. This way, if we add more nodes to the cluster the clients become aware of the full topology dynamically. Awesome stuff!

Step 1

We will first need to create an overlay network for the cluster, here named nats-cluster-example , along with an initial NATS server using latest Docker:

docker network create --driver overlay nats-cluster-example docker service create --network nats-cluster-example \ --name nats-cluster-node-1 nats:0.9.4 -DV

Step 2

Next, we confirm which node of the Docker Swarm NATS Server ended up running on, and confirm its IP:

docker service ps nats-cluster-node-1 ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR b81qv4ljs7g7b52vmnlojktug nats-cluster-node-1.1 nats:0.9.4 node-2 Running Running 32 seconds ago docker network inspect nats-cluster-example

[ { "Name" : "nats-cluster-example" , "Id" : "944z42rg2hjwgsv9xubmomvi5" , "Scope" : "swarm" , "Driver" : "overlay" , "EnableIPv6" : false , "IPAM" : { "Driver" : "default" , "Options" : null , "Config" : [ { "Subnet" : "10.0.1.0/24" , "Gateway" : "10.0.1.1" } ] }, "Internal" : false , "Containers" : { "e0d105ed7703aa5de939861f75087671c96128fbe799bf6ba38cd15c55aaef07" : { "Name" : "nats-cluster-node-1.1.0j0wde7hbqn735eqgpslkxowf" , "EndpointID" : "e798782eaac9bbd36decf3c6e5defd56faa88a3c7a09df608ffd1f2ce1969ed8" , "MacAddress" : "02:42:0a:00:01:03" , "IPv4Address" : "10.0.1.3/24" , "IPv6Address" : "" } }, "Options" : { "com.docker.network.driver.overlay.vxlanid_list" : "258" }, "Labels" : {} } ]

On node-2 of the Docker Swarm, notice that the NATS server is now running:

docker logs e0b4d7b2f7f3 [1] 2016/08/15 11:31:41.680139 [INF] Starting nats-server version 0.9.4 [1] 2016/08/15 11:31:41.680217 [DBG] Go build version go1.6.3 [1] 2016/08/15 11:31:41.680259 [INF] Starting http monitor on 0.0.0.0:8222 [1] 2016/08/15 11:31:41.680348 [INF] Listening for client connections on 0.0.0.0:4222 [1] 2016/08/15 11:31:41.680377 [DBG] Server id is JngwpOevXl1rhAovFO8Su6 [1] 2016/08/15 11:31:41.680386 [INF] Server is ready [1] 2016/08/15 11:31:41.680731 [INF] Listening for route connections on 0.0.0.0:6222

Step 3

Next, we will create another service which connects to this server within the overlay network; note that we have an initial IP for connecting to the server:

docker service create -- name ruby - nats -- network nats - cluster - example wallyqs /ruby-nats:ruby-2.3.1-nats-v0.8.0 -e ' NATS.on_error do |e| puts "ERROR: #{ e } " end NATS.start(:servers => ["nats://10.0.1.3:4222"]) do |nc| inbox = NATS.create_inbox puts "[ #{ Time . now } ] Connected to NATS at #{ nc . connected_server } , inbox: #{ inbox } " nc.subscribe(inbox) do |msg, reply| puts "[ #{ Time . now } ] Received reply - #{ msg } " end nc.subscribe("hello") do |msg, reply| next if reply == inbox puts "[ #{ Time . now } ] Received greeting - #{ msg } - #{ reply } " nc.publish(reply, "world") end EM.add_periodic_timer(1) do puts "[ #{ Time . now } ] Saying hi (servers in pool: #{ nc . server_pool } " nc.publish("hello", "hi", inbox) end end '

Step 4

Now we are ready to add more nodes to the cluster via more docker services :

docker service create --network nats-cluster-example \ --name nats-cluster-node-2 nats:0.9.4 -DV -cluster nats://0.0.0.0:6222 -routes nats://10.0.1.3:6222

Add in more replicas of the subscriber:

docker service scale ruby-nats=3

Then confirm the distribution on our Docker Swarm cluster:

docker service ps ruby-nats ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR 25skxso8honyhuznu15e4989m ruby-nats.1 wallyqs/ruby-nats:ruby-2.3.1-nats-v0.8.0 node-1 Running Running 2 minutes ago 0017lut0u3wj153yvp0uxr8yo ruby-nats.2 wallyqs/ruby-nats:ruby-2.3.1-nats-v0.8.0 node-1 Running Running 2 minutes ago 2sxl8rw6vm99x622efbdmkb96 ruby-nats.3 wallyqs/ruby-nats:ruby-2.3.1-nats-v0.8.0 node-2 Running Running 2 minutes ago

The sample output after adding more NATS server nodes to the cluster, is below - and notice that the client is dynamically aware of more nodes being part of the cluster via auto discovery!:

[2016-08-15 12:51:52 +0000] Saying hi (servers in pool: [{:uri=>#<URI::Generic nats://10.0.1.3:4222>, :was_connected=>true, :reconnect_attempts=>0}] [2016-08-15 12:51:53 +0000] Saying hi (servers in pool: [{:uri=>#<URI::Generic nats://10.0.1.3:4222>, :was_connected=>true, :reconnect_attempts=>0}] [2016-08-15 12:51:54 +0000] Saying hi (servers in pool: [{:uri=>#<URI::Generic nats://10.0.1.3:4222>, :was_connected=>true, :reconnect_attempts=>0}] [2016-08-15 12:51:55 +0000] Saying hi (servers in pool: [{:uri=>#<URI::Generic nats://10.0.1.3:4222>, :was_connected=>true, :reconnect_attempts=>0}, {:uri=>#<URI::Generic nats://10.0.1.7:4222>, :reconnect_attempts=>0}, {:uri=>#<URI::Generic nats://10.0.1.6:4222>, :reconnect_attempts=>0}]

Sample output after adding more workers which can reply back (since ignoring own responses):

[2016-08-15 16:06:26 +0000] Received reply - world [2016-08-15 16:06:26 +0000] Received reply - world [2016-08-15 16:06:27 +0000] Received greeting - hi - _INBOX.b8d8c01753d78e562e4dc561f1 [2016-08-15 16:06:27 +0000] Received greeting - hi - _INBOX.4c35d18701979f8c8ed7e5f6ea

Conclusion

Built-in Docker Swarm mode in Docker Engine can be very handy for deploying a distributed system where NATS is being used for handling the internal communication among components via an overlay network, and external communication is being exposed through a load balancer. This in combination with the cluster discovery features from latest NATS releases make microservices operations a breeze, and incredibly flexible and scalable.

Want to get involved in the NATS Community and learn more? We would be happy to hear from you, and answer any questions you may have!

Follow us on Twitter: @nats_io