Designing an automated warehouse Watch Now

The growth of interest in Kubernetes coincides with a pivot point. Enterprises would like to make application deployment work just like firing up a cluster on AWS EC2. Kubernetes has become the de-facto standard for orchestrating all those Docker containers, providing the hope to enterprises that they too could gain the flexibility and rapid, codeless deployment environments that public cloud IaaS services provide.

For the database folks, however, there's been a major roadblock with Kubernetes, as it was designed for a world of application services that don't require state. Databases require state as a key requirement for ensuring that commits don't get corrupted by disruptions -- a basic requirement of ACID. Typically, databases handle their own state management, but when there are external dependencies, such as with additional application tiers or disjoint services, typically an external broker has to pick up where the database leaves off.

We've been there before. That was the original case behind appservers, which added the state that Java could not guarantee. And for container orchestration, the Kubernetes project has responded with its own primitive, StatefulSets. But the problem with that approach is that it is, well, primitive.

Couchbase, which has been preparing for this day by rearchitecting the database for microservices, is announcing at KubeCon in Barcelona this week, a new "autonomous operator" for Kubernetes. The operable notion, so to speak, is easing the process by which clusters are configurated and fired up or wound down.

With the Couchbase autonomous Kubernetes operator, the system admin or developer simply writes a YAML configuration file that specifies the requirements for instances and all the related dependencies. There is also Helm chart support for helping you define the Kubernetes workflows. And once in operation, the operator can detect degraded clusters that can help support a high availability/failover strategy. This will work on any cluster supporting Kubernetes, both in public and private clouds.

For enterprises, this marks the beginning of the journey that very much resembles the original migration to the cloud, which started with test/development clusters before the cloud was taken seriously for running production workflows.

Without Kubernetes, the database would have to manually install a package on each node, activating the right ports make connections to the right nodes, configurate the load balancers, installing the necessary certificates for supporting SSL encryption, and setting up the firewalls. And if you wanted to dynamically add resources to a cluster, you'd have to run the whole process all over again. No wonder that relatively few enterprises have been able to master their own private or hybrid clouds.

This is still early days for enterprises to build their Kubernetes environments. But for Couchbase customers, the opportunity to take advantage of modern deployment techniques means more than buzzword compliance, as most Couchbase customers currently run their workloads in the cloud.