Key Takeaways Kubernetes has become the de facto standard for container orchestration, and many organisations run multiples clusters. Communication within clusters is a solved issue, but communication across clusters requires more design and operational overhead.

Before deciding on whether to implement multicluster support, you should understand your communication use case.

You should also identify what you want from a solution (single pane of glass observability, unified trust domain etc), and then create a plan on how you implement this.

There are several multicluster service mesh approaches, such as common management, cluster-aware service Routing through gateways, flat network and split-horizon endpoints discovery service (EDS).

Istio has existing multicluster support, additional new functionality in 1.1, and even more appearing in the future.

Kubernetes has become the de facto standard for container orchestration in the enterprise. And there’s good reason for it — it provides a slew of capabilities that makes managing containerized applications much easier. Kubernetes also creates some new challenges, a primary one being the need to deploy and manage multiple Kubernetes clusters in order to effectively manage large-scale distributed systems.

Imagine you've already got an app designed and coded, and you've built containers — you just need to run them. Getting from code to running app is exhilarating, but as anyone who has built a containerized application knows, it’s not nearly as simple as it might appear at first glance. Before deploying to production, there are various dev/test/stage cycles. Additionally, there's a scaling aspect — your production application may need to run in many different places, for reasons like horizontal scalability, resiliency, or close proximity to end users.

More environments, more (cluster) problems

You typically don't get far before even a simple greenfield app concept ends up requiring multiple deployment environments. If you're migrating an existing application, you’re bound to run into even more challenges like different security domains, different organizations/billing, and affinities for one cloud vendor's machine-learning toolkit.

The most common approach to solving this problem is to create multiple Kubernetes clusters, each one dedicated to running your app components in its particular environment. In a high-security domain, you will make extensive use of Kubernetes role-based access control (RBAC) and have auditing capabilities. The test environment should reproduce a lot of production behaviors but be tailored for easy debugging and inspection. For your dev environment... — well, maybe you're like me and you’ll just open up the Docker preferences and check the Kubernetes box. Ease of use and ephemerality are the name of the game here.

You’ll likely end up with at least a few Kubernetes clusters, each cluster hosting microservices. The communication between these microservices in a cluster can be enhanced by a service mesh. Inside of a cluster, Istio provides common communication patterns to improve resiliency, security and observability. What about between and across clusters?

Operating multiple Kubernetes clusters doesn’t have to be scary, but running multiple clusters does require you to consider how they will communicate and interact in order to easily deliver the great apps that run on top. A service mesh like Istio can make multicluster communication painless. Istio has multiclustersupport, added new functionality in 1.1, and plans to add even more in the future. Teams should think about employing a service mesh to simplify communication across multiple clusters as well.

Common use cases

At Aspen Mesh, we get asked about running a multicluster service mesh most commonly for these user needs:

I have multiple clusters because of my organization size, and I want one place to see and manage them. My clusters don't generally do inter-cluster traffic, or when they do, it's through well-defined APIs. I have multiple clusters for high availability; they are clones of each other and it's very important that if one cluster fails, the other cluster can take over. I have multiple clusters that combine to form a higher-level app. Microservices in one of those clusters need to communicate with microservices in another to provide the proper end-to-end app experience.

It's the third category where multicluster requires inter-cluster traffic. If you want inter-cluster traffic support, your implementation will depend on the networking between the clusters, and on your requirements for fault tolerance.

What you can get out of multicluster

When you're thinking about multicluster and service mesh, you should start by identifying what you want, then shift to how you get it.

Single pane of glass

Your multiple service meshes (as many as one for each cluster) operate from one place. You can view the config, metrics, and traces for all clusters in a single pane of glass.

Unified trust domain

You use a service mesh to provide workload identification, protected by strong mutual-TLS encryption. This zero-trust model is better than trusting workloads based on topological information like source IP: you're relying on cryptographic proof of what they are, not a fragile perimeter stack to control where they came from.

A unified trust domain means that all the workloads can authenticate each other (what they are) by tying back to a common root of trust. The service-mesh control planes are each configured for that common root of trust, whether there are one or several of these planes.

Independent fault domains

A cluster that does not rely on another cluster for proper operation of the cluster and associated infrastructure itself is an independent fault domain. I am including service mesh as associated infrastructure — if you're installing a service mesh, you are doing it to move communication resiliency to an infrastructure layer underneath the application. If a failure of the service mesh in one cluster can break the service mesh in another cluster, it doesn't qualify as an independent fault domain.

Intercluster traffic

You need traffic between clusters to remain part of the service mesh if you want services in one cluster to communicate directly with services in another, and you want that communication to have the benefits of a service mesh, like advanced routing, observability, or transparent encryption. In other words, you want your east/west traffic to leave one cluster, transit some intermediate network like the Internet, and then enter another cluster.

This is probably the first thought for most people when they think about a multicluster service mesh, but I'm calling it out separately here because it has implications for fault tolerance.

Heterogeneous/non-flat network

A non-flat network supports services across multiple clusters, without flat networking requirements. This means you can do things like allocate IPs in one mesh without consideration for another, and you don't need VPNs or network tunnels for communication across meshes.

If your organization has already created a bunch of different clusters without de-conflicting pod IP-address ranges or you just don't ever want to enter that morass again, this will be an attractive property to you.

Multicluster service-mesh approaches

Having laid out the different properties you might be looking for out of multicluster, I can walk through what various approaches deliver.

Independent clusters

This is the un-multicluster. Just because you have multiple clusters and each uses a service mesh doesn't mean you must adopt a unifying multicluster service mesh. Ask yourself why you ended up with multiple clusters in the first place. If you want each cluster to be its own independent fault domain, it makes sense to isolate and remove cross-cluster dependencies. If that meets your needs, there's no harm in treating a service mesh as another per-cluster service like pod scheduling or persistent disk management.

Common management

A step above the independent-clusters approach is a common management system for multiple clusters. In this model, each cluster operates independently but you manage the set of meshes via a common management interface. It's good design to have the thing you use to monitor and debug your system (or, in this case, systems) reside outside of the system itself so when the system is broken, you can still inspect and fix it.

If you choose to use a common root of trust (certificate authority or signing certificate) across these clusters then you can also have a unified trust domain.

This is a good choice if independent fault domains is a top priority. This option is a good fit for consuming software as a service because you can get one external pane of glass to unify everything, backed by a service-level agreement.

We made sure Aspen Mesh supports this use case even if you don't enable any of the inter-cluster approaches because we see value in this resiliency. This approach supports the others, too, but we're careful not to rush our users into inter-cluster traffic when it's not required.

Cluster-aware service routing through gateways

This approach in Istio involves multiple independent service meshes, one in each cluster, and some configuration trickery to allow services in one cluster to communicate with services in another cluster. You start by creating one unified trust domain for all your meshes. Next, you configure an ingress gateway to accept trusted traffic that comes from a service in another peer cluster. Finally, you configure service entries to allow traffic for certain services to be routed out of one cluster and sent to another.

This is the first method that allows services in different clusters to communicate directly with each other. Also, each cluster is still an independent mesh control plane and fault domain. This means that if the service mesh in cluster B fails, cluster A will still work; it'll just look like the services in cluster B are unavailable. The burden of configuring this cross-cluster traffic is placed on the user.

Flat network

This model dictates a service mesh across all your clusters. You arrange it so that the pods in each cluster have non-overlapping IP addresses, so any pod can route traffic to any other pod in any cluster. You might have a bunch of clusters behind a common firewall or you might build VPN tunnels between clusters. You configure your service mesh to combine the discovered pods, services, and configs from each cluster into one overall view.

A flat network makes it appear as if you have one super service mesh that spans all your clusters. There are some downsides. This super service mesh is managed by one control plane, so if it has problems, the service mesh in all clusters will have problems. If you originally partitioned into multiple Kubernetes clusters for fault tolerance, this approach negates that. Another consideration is that the control plane has to scale to manage all clusters. And you have to make this flat network perform well enough to handle the control plane and cross-cluster traffic.

Split-horizon Endpoints Discovery Service (EDS)

This approach also creates one service mesh across clusters but does not require flat networking. You still have one control plane that discovers pods, services, and configs from each cluster but Istio’s EDS, which has functionality akin to split-horizon DNS, replaces the requirement for a flat network.

The sidecar for a pod in one cluster is configured with a list of endpoints for each service it wants to talk to. If a pod is in the same cluster, it shows up directly in the EDS list. If a pod is in another cluster, an ingress gateway for the other cluster appears instead. The pod chooses an endpoint to talk to and sends traffic — if the endpoint is local, the communication is direct pod to pod. If the pod chooses a remote endpoint, it sends traffic to the address of the relevant ingress gateway, marked for the service the pod wants to talk to. The ingress gateway takes the traffic and sends it to one of the pods in its cluster that implements the service. The ingress gateway uses Server Name Indication (SNI) to know where the traffic is destined.

Like the flat-network approach, this creates a unified service-mesh control plane and adds a single fault domain and single trust domain. It does not require a flat network, it only requires that one cluster can send traffic to the public address of ingress gateways for the other clusters.

To multicluster or not to multicluster

If you are going to be running multiple clusters for dev and organizational reasons, it’s important to understand your requirements and decide whether you want to connect these in a multicluster environment and, if so, to understand various approaches and associated tradeoffs with each option.

If you have read this far, you probably have decided to multicluster. The real question is what’s the best method to implement. Hopefully, the table below will help you to decide on the right approach for you.

Unified Management Unified Trust Heterogeneous Network Independent Fault Domain Inter-cluster traffic Independent ✓ ✓ Common Management ✓ ✓ ✓ Flat Network ✓ ✓ ✓ Split Horizon ✓ ✓ ✓ ✓ Cluster-aware Service Routing ✓ ✓ ✓ ✓

A service mesh-like Istio can help, and when used properly can make multicluster communication painless. If you’d like to talk more about my take on why and how teams should think about employing a service mesh to simplify communication across multiple clusters. multicluster communication with a service mesh, or a service mesh at all, feel free to reach out to me at andrew@aspenmesh.io.

About the Author

Andrew Jenkins is the CTO at Aspen Mesh, where he’s building an enterprise service mesh to help organizations take the burden out of managing microservices. A software and network architect for container environments like Kubernetes, Jenkins has a history of technical leadership driving fast-moving teams toward tangible outcomes. His expertise includes software development in C++, JavaScript (Node.js), Python, C, Go, and Java. Jenkins also has experience in software and hardware testing, FPGAs, and board design for space scientific instruments.