Right now every container orchestration engine needs its own storage driver. That's senseless, so major companies are working together to build a standard.

Containers can boost cloud-native apps and boost automation Watch Now

Dell-EMC, Red Hat, and others are progressing in their new method for connecting storage to container organizer software.

Currently, every orchestration engine--the industry term for software that arranges resources between application containers and hardware--has its own way of communicating with storage. That's unnecessarily complicated, but a proposed standard called Container Storage Interface (CSI) evolved in February 2017 to solve the problem and now has support from people involved with popular engines such as Docker Swarm, Google Kubernetes, and Apache Mesos.

SEE: Cloud data storage policy (Tech Pro Research)

Container technology goes back to the 1970s, but the storage problem is newly prominent because of choices in hardware and application development that didn't exist several decades ago. For example, most storage devices can only connect to one container at a time, but there are modern applications such as cluster locking and replication where a solution might be necessary, explained Dell's Josh Bernstein, vice president of technology for open-source software.

As such, "CSI is really a community-based effort," Bernstein said. In addition to Dell and Red Hat, the CSI committee also has support from Google, Hitachi Data Systems, IBM, Mesosphere, and NetApp.

Dell this month released a new version of its storage orchestrator called Rex-Ray. The new version is the first software to use CSI. Bernstein acknowledged that there won't be a real-world use of the new function until container orchestrators also adopt the same connection, because right now there's nothing for Rex-Ray to connect. It's a chicken-and-egg problem and Dell decided to go first, he explained.

"I think with the release last week it is there and it is available," Bernstein said, referring to CSI's current level of usefulness. However, "We're not calling it a standard just yet because standard implies there's a wide adoption and we're not quite there," he noted. Future directions could include a simpler container shutdown process and resource prioritization, he said.

Red Hat, known for its enterprise version of Linux, also distributes an enterprise Kubernetes orchestrator called OpenShift. The company supports CSI because it believes storage management should remain with storage systems, said Stephen Watt, chief architect for emerging technologies. A discussion to put CSI support in Kubernetes is likely to occur at a meeting in October 2017, which may supersede the existing Kubernetes FlexVolume method, he said.

SEE: Special report: Riding the DevOps revolution (free PDF) (TechRepublic)

Jesse White, a consultant specializing in Docker and leader of the DockerNYC meetup, said he likes the idea of a Container Storage Interface. "That makes sense," he said. "For about four years now it's been a question of security, storage, and networking. They're always the hard things with containers, and it really hasn't changed. Storage and stateful services are still really hard."

"I work with large enterprises generally," White continued. Despite the recent industry focus on speed, such as with flash arrays and non-volatile memory connections, "Most of them want to have reliable interfaces. They want storage that they can depend on, that's not necessarily the fastest in the world."

Data Center Trends Newsletter DevOps, virtualization, the hybrid cloud, storage, and operational efficiency are just some of the data center topics we'll highlight. Delivered Mondays and Wednesdays Sign up today

Also see