The twin goals of the software defined data center (SDDC) are to foster greater agility in both infrastructure and architecture and to boost the speed at which raw data is converted into usable knowledge. But even though the SDDC lifts the three basic elements of data infrastructure – compute, storage and networking – off of the hardware layer and onto abstract software constructs, the ideal of a perfectly aligned, eminently harmonious data ecosystem remains elusive. The fact is that software alone does not eliminate the data bottlenecks, resource contention issues, silo architectures and other artifacts that inhibit the smooth flow of data.

As in the hardware world, the greatest contributor to latency in the SDDC is getting data in and out of storage resources. And while it may seem like this is strictly a networking problem it is actually a shared responsibility driven by poor coordination between virtualized compute resources and their concomitant elements on the storage and network layers.

When SDS Meets SDN

Because both software defined storage (SDS) and software defined networking (SDN) are newcomers to the enterprise, it shouldn’t come as a big surprise that most of the performance issues lie at the intersection of these two elements. According to technology consultant Jim O’Reilly, the storage/network relationship is even more complicated in virtual environments than in physical ones due to the increased prevalence of heterogeneous architectures. When anyone with a modicum of skill can provision their own storage and networking, it shouldn’t come as a big surprise that those architectures will start to reflect the varying needs of applications and users, not IT’s management requirements.

Most SDDCs will gravitate toward converged compute and storage resources, since this is one of the best ways to reduce latency and backbone traffic. But if this fact is not reflected in administrative tools and emerging automated orchestration stacks, key workloads may suffer because they are not gaining the instant access to the right resources.

A prime example, O’Reilly notes, is an app that may suddenly decide to render a large video file that can only be handled by a GPU. Orchestration may be able to provision the instances, but not necessarily in the same rack as the rest of the app’s data. That means a wide-lane LAN will be needed to make the connection and then storage has to be informed as to the available bandwidth and how it is to be allocated between normal data the video rendering process.

This may not seem like a big deal, but as speed and complexity increase the enterprise will need to continually fine-tune its orchestration and management policies to reflect the changing environment.

The Software Controller

Much of this activity will take place on the software-defined storage controller, which is emerging as one of the key technology development initiatives of the coming decade, according to Persistence Market Research. The SDS controller is literally tasked with ensuring that resources are functioning efficiently and with sufficient scale to enable the smooth flow of data.

The danger, however, is that organizations do not solve existing network issues without creating new ones. Since it relies on proper programming to carry out its mandate rather than hands-on management, the SDS can suffer from the same problems that plague any other piece of software: poor runtime execution, glitches in code, update inconsistencies and the like. This is the main reason why many organizations are planning their SDS infrastructure around dual controllers, or even more.

At the moment, though, the biggest challenge facing the SDS controller is the abundance of development projects currently underway. While proprietary systems are common in any technology, the process of setting SDS standards is still at a very rudimentary stage and the field is rife with non-uniform approaches and multiple SDS data structures. The closest the storage industry has to an SDS standard is the Linux Foundation’s OpenSDS project, but even within this general framework there are dozens of individual initiatives that will most likely provide commonality in some areas but not in others. As the SDDC reaches past the private cloud and into public/hybrid architectures, a high degree of compatibility across disparate resources will be necessary if the enterprise hopes to build broad flexibility into its provisioning and resource allocation policies.

When it comes to increasing storage speed, of course, nothing beats a good, old-fashioned boost to I/O. This is what an open community called the Fast Data Project, an offshoot of the Linux Foundation, claims to have accomplished with help from Intel’s new Xeon Scalable processor. With the release of the new version of its FD.io (Fido) vSwitch, the company says it can double packet throughput to the terabyte level, enabling high-speed exchanges across software-defined infrastructure.

The FD.io architecture is built around Vector Packet Processing (VPP), which leverages tools like vector instruction, cache optimization and packet pre-fetching to boost performance to I/O-connected systems using the DPDK plug-in. In recent tests, the system showed an aggregate forwarding rate of 948 Gbps of Layer 2/3 traffic even as advances in the CPU microarchitecture decreased the number of cycles per packet. And the best news for developers is that the software can take advantage of the new Xeon chips without modification.

Even Faster Solid State

And it probably doesn’t need to be said that when we are talking about boosting storage speeds to suit software-defined architectures we aren’t referring to either disk or tape. But it won’t just be the presence of solid-state storage that provides the greatest benefit to software-defined architectures, but in the support systems around it.

For example, applying traditional block and file storage to a solid-state array is effective when data loads consist of relatively few but rather large segments of structured data, says Lenovo’s Sumir Bhatia. But when workloads start to transition to millions, perhaps billions or even trillions of small files, as will likely happen in emerging IoT and Big Data environments, it is better to go with object storage.

As well, emerging interface protocols like NVME are poised to take solid-state media to an entirely new performance levels compared to existing SAS and SATA platforms. NVMe offers lower latency, higher bandwidth and less infrastructure overhead (no more HBAs) that make it ideal for modular, converged infrastructure and ultra-high-speed storage formats like 3DXP.

The SDDC will undoubtedly introduce new levels of performance and new data-driven opportunities for the enterprise, but it will also require a radically different approach to resource management than what exists today. While it is true that many of the daily operational problems that hamper data productivity are easier to solve, or circumvent, with a fungible, federated architecture, particularly once intelligent automation and orchestration are added to the mix, the enterprise will still have a job on its hands defining the parameters of its data ecosystem and developing the policies governing data access, usage, sharing and myriad other functions.

It may not be digital nirvana, but the SDDC will certainly propel the enterprise to an entirely new level of data productivity.