Why a Service Mesh is not a replacement of API Management

As a disclaimer, I work at Red Hat, more specifically, at the 3scale Team (acquired 2 years ago) developing the 3scale API Management Solution.

Lately, a question is popping up more and more when talking with our customers:

“Why do I need API Management if I use Istio?”

To answer this question, first, we need to understand what is what, but if you want a spoiler: 3scale API Management and Istio are amazing together.

I will try to describe what problems those two solutions aim to solve, focusing on 3scale API Management and Istio Service Mesh (those are the two I know better).

What is an API Management solution?

If we take a look at the Wikipedia:

“API management is the process of creating and publishing web APIs, enforcing their usage policies, controlling access, nurturing the subscriber community, collecting and analyzing usage statistics, and reporting on performance.”

This is a good definition. As a company that has created some kind of internal service, I now want to build a business around it by providing an API to external subscribers. Of course, I want to monetize it by offering several subscription plans with different usage limits, scopes, and be able to invoice my customers automatically.

Additionally, external Developers should find my API easily, and sign up for a plan in a self-service way with just their credit card and all of this should be transparent for my API code…

If we analyse those requirements, we can fit them into the next categories:

Access Control and Security : Control who can access my API and how.

: Control who can access my API and how. API Contracts & Rate limits : How many requests can a User do based on a subscription.

: How many requests can a User do based on a subscription. Analytics and Reporting : How your API is doing? Which methods are the most used? Any error? which are the top/trending APIs?

: How your API is doing? Which methods are the most used? Any error? which are the top/trending APIs? Developer Portal, documentation : Let Developer find your API and sign up for a subscription plan

: Let Developer find your API and sign up for a subscription plan Billing: Send the invoices and charge developers.

How does an API management solution enforces all of this? Thanks to a key component called the API Gateway.

This is a component that sits in the middle of the traffic flow, so clients’ requests go through it, protecting your API endpoints, and communicating with the other API Management components to let a user access your API or not.

This is usually achieved by authenticating and “rate limiting” the user requests, imagine this scenario:

User A is subscribed to the “Basic” plan

is subscribed to the plan “Basic” plan defines some limits for several API operations (HTTP Method + HTTP path), for example: “GET /products” and “POST /shipments”

(HTTP Method + HTTP path), for example: “GET /products” and “POST /shipments” Those Limits can be defined, per second, minute, month…

“GET /Products” is limited up to 10 requests per minute.

In this case, when the user A tries to make more than 10 requests in a minute, those extra request will get rate limited with 429 (Too many requests), or if the authentication is not valid, the user will get a 403 (Forbidden), authentication can be provided as part of OAuth, some query param or a header.

Rate limits are a key part of API Management. That’s how business rules are enforced to the end user by the API Gateway but is necessary to understand that rate limits can be extraordinarily complex and based on multiple rules or simple, like per client IP. The ability to define really complex rate limit scenarios (business rules) is what makes API management so powerful.

So, let me say something out loud:

API Management is NOT (only) Rate limiting.

What is a Service Mesh?

Previously, we talked about APIs, we didn’t talk about services, applications, ports, connections, retries… because at the API Management level, we don’t really care about that, but we will now.

What’s really behind your awesome API? Multiple services talking to each other, interacting between them, to form a single big API and each one written in a different programming language, maintained and operated by a different team in various places in the world inside a big organization. Does this ring a bell? Yeah! Microservices!

Microservices. Or connected dots.

So we have our big API, composed of multiple of services, this sounds really nice, but more and more teams are pushing new services for new features and/or new requirements, and at some point in time, questions and problems due to the growing operational complexity of your architecture will popup:

What happens if any of those internal requests between services fail

Where are my requests failing ?

? Why this API endpoint is so slow ? Which service is at fault?

? That service is really prone to errors… Can we retry those calls if they fail?

if they fail? Someone is hammering this service every day at the same time, we should add some rate limit to avoid that!

to avoid that! That service shouldn’t be able to contact that other service…

Those questions/problems are the ones that a Service Mesh can help solve. And fall into those category types:

Resiliency : Timeouts, Retries, Circuit Breakers, fault handling, load balancing

: Timeouts, Retries, Circuit Breakers, fault handling, load balancing Rate limiting : Infrastructure rate limiting based on multiple sources.

: Infrastructure rate limiting based on multiple sources. Traffic routing : Routing capabilities by path, host, header, cookie base, source service…

: Routing capabilities by path, host, header, cookie base, source service… Observability : Metrics, logging, distributed tracing

: Metrics, logging, distributed tracing Security: mTLS, RBAC…

And all of this enforced in a transparent way for the application.

Let's see how Istio works:

Istio Components diagram

Istio uses the "sidecar container" pattern, extending the functionality of the “main” container with a second one running within the same POD. The “Main” container is our application, the sidecar container, is the Istio proxy, this is based on "envoy".

Once injected, all the traffic going from (and to) our main container, will be "hijacked" by the proxy (Using Iptables magic) as you can see in the previous diagram.

This allows Istio to control the traffic and report what's going on to the control plane. More specifically to Mixer, the telemetry and policy engine.

What do they have in common?

We have defined both technologies, what do you see in common? They try to solve different problems, but using some similar techniques…

Those two solutions have a thing in common:

But it's important to remember that the rate limits are used to enforce different things: Business rules or Infrastructure limits.

So, they are not mutually exclusive, we should think about them as different layers in your infrastructure:

API Management: Handles access to your API based on developers, subscription plans, applications, invoices… Service mesh: Makes your API services secure, easy to monitor, and resilent.

How we are combining 3scale API Management and Istio Service mesh ?

Keep tuned for a series of more technical posts about how 3scale is adding full API Management capabilities to the Istio Service Mesh either by using our API Gateway APIcast or natively extending Istio using the 3scale Istio Adapter.