How NOT to be API-misled for connectivity!

It was a few years back when API-led connectivity was getting popular and our customers wanted to see it being brought into practice. Some of them also began to choose their hybrid integration products based on the product’s inherent support for the approach. After having implemented two large hybrid integration programs for two different industries (Banking and Manufacturing), I thought I should share my experience as do’s and don’ts for API-led connectivity and Hybrid Integration.

API-led connectivity

The approach was termed as the next step in the evolution of SOA, which is why its principles and the very fundamental concepts will remain timeless, at least in the context of software architecture.

Unlike SOA, for API-led connectivity, the timing was right. What makes it pertinent, feasible, and successful?

Cloud Adoption — Enterprises started to adopt SaaS applications and also began moving their enterprise applications to the cloud. What did that mean for integration and connectivity? Integration components can no longer remain as monolith components deployed on heavy on-premise runtimes. But had to be broken down as multiple components with well-designed interfaces that can be deployed to lightweight hybrid runtimes. For e.g., If your System of Record (SoR) was on the cloud, you could deploy your system API to the “cloud” runtime of your hybrid integration platform. While if your SoR was still on-premise, you deploy the system API to the on-premise runtime component of your hybrid integration platform True Hybrid Integration Platforms API led as an approach was made feasible only when integration platforms began to evolve as true hybrid integration platforms. iPaaS (Cloud Offering), or the heritage on-premise integration product that evolved to support SaaS integration/API gateway, was not enough. You needed a platform that truly supported your pre and post-digital era integration needs while your center of gravity was still “on-premise” but your destination is the Cloud. There were not many then, and even now, platforms that support true hybrid integration. Lesson Learned from EAI, SOA, and API — And what makes it successful now is perhaps the years of implementation experience from traditional EAI, SOA, and API projects.

Where you do NOT want to be misled

API-led connectivity seems to be a natural fit, especially for industries like banking and others where connectivity patterns are primarily “pull” patterns. For example, unlocking enterprise data/information — channel applications need to pull data from SoR.

However, for other industries such as manufacturing, integration patterns are primarily “push” patterns. For example, a global ERP instance pushes out millions of Service Orders per day and every such event in the life cycle of the service order requires critical actions to be performed in multiple applications both SaaS and on-premise applications.

This is where it can get very “creative” about how some of the integration architects tend to overdo the API-Led connectivity approach.

HTTP(S) is perhaps mistaken to be the only implementation protocol for API-Led Connectivity. Also perhaps due to the fact that most hybrid integration platform vendors supported only HTTP(S) as the option for inter API component communication. Basically, your System, Process, and Experience API cannot always remain on pictures and must get implemented. And they do get implemented, they are APIs deployed to runtimes and are invoked by HTTP methods. Back in the EAI days, the principles were the same — well-defined interfaces, loose coupling, etc. but the protocol used for inter “API” communication could be native/proprietary as the integration landscape for mostly on-premise. Then, with SOA, those principles still remain relevant, but with web services becoming the popular implementation of SOA, SOAP/HTTP(S), or JMS became the implementation protocol. Now with REST APIs, and cloud adoption, HTTP becomes a natural choice, as it is the default protocol of the world wide web. The tendency to convert every integration pattern as a system, process, and experience API. Several other integration patterns such as event-driven or file integration patterns barely lend themselves to the API-led approach. Even if the business objects are well-designed using the API-led approach and each layer offers a system, process, and experience API, for those APIs, if there are no clients to “pull” or use that API, the approach has not proved useful.



Basically, some of the use cases are “Event-Driven.” For example, Mater Data Synchronization and those need to be treated differently that best suits the needs/constraints of the pattern. In another scenario, what is inherently a file transfer pattern in a hybrid scenario is redesigned to follow an API-Led approach. Worse, when large files are streamed over HTTP(S) from cloud to on-premise, some of the products support inter API communication only through HTTP(s). The tendency to force-fit a System API for an SoR that imposes constraints such as scalability to support a large number of API calls. Or if the billing model is based on the number of APIs invoked. Or the Systems of Record functions primarily as a publisher of events.

Conclusion

As hybrid integration architects, you may want to consider an API-led approach as one of the primary approaches, but not as the only approach in Hybrid integration. Event-driven and file integration patterns, for example, will still need a different approach.

As customers evaluating hybrid integration platforms and not purely API Capabilities, you may want to look for how well the product lends itself to an API-Led approach, but at the same time, not losing sight of features needed for hybrid integration.

For example, Traditional Partner Integration Patterns with support for large file transfers across partner networks — does the hybrid integration platform support secure MFT service?

For example, event driven patterns — does the hybrid integration platform support a reliable event streaming component such as Kafka?