Post-Flash Storage gets real, software-defined storage gains real traction. 25/50/100GE, AFA, ARM, NVM, HCI, SDS and SMR on the rise

25/50/100G/bit Ethernet

First of the storage trends is for 25/50/100 Gigabit Ethernet. The first products have just reached general availability, so in 2016 we will see wide adoption of these. For our segment 25 Gbps Ethernet will be especially popular because it is a single-lane technology (like 10G, not like 40G), It is backward compatible with 10G (uses the same SFP-type connector) and has a low cost for NICs, cables, and switches. By the end of the year the cost difference between 10 and 25 will be minuscule, so the choice of 25G should be obvious for any new deployment during 2016.

All-Flash Arrays (AFA)

All-flash memory has been gaining market share during the last few years. In 2016 even late adopters will switch to Flash for their primary storage needs. It still has to go through a row or price reduction to become affordable.

ARM Servers

ARM servers are coming up with players like Broadcom. AMD and Samsung are also going to try their chances of gaining (share) of the ARM server chip market. We see ARMs expanding their share in the Data Centers in 2016 because of their lower power consumption and higher density. For the same reasons they are a good fit for software-defined storage solutions and will allow optimizing total cost even further.

Non-Volatile Memory (NVM)

NVM is a new class of storage media. It is faster than (NAND) flash, slower than RAM, but it is non-volatile. We expect to see first productized 3D XPoint memory by Intel/Micron. Also, HP/SanDisk (now WD) announced cooperation around memristors but it is unsure if we’ll see a product in 2016. The first product will be a 3D XPoint PCIe card to be released in Q1 2016.

In the beginning, prices will be higher than the ones of NAND flash, but in the long run, non-volatile memory can replace flash entirely. As a result, NAND flash is going to become the new disk, while NVM will take the role of flash. This means a new crop of storage vendors focusing on the new storage tier as it happened with NAND flash. We expect the first players to be announced in 2016.

Hyper-Converged Infrastructure (HCI)

Hyper-converged solutions are going to get wider adoption as they reduce complexity. However, they are still costly and by concept are best suited for smaller deployments of 2-5 nodes and thus for the SMB/ROBO segment. At a larger scale, HCI is not the solution of choice for cost and manageability reasons.

Software-Defined Storage (SDS)

SDS has been in the storage trends and hot topic during the last couple of years, however, the majority of deployments were PoC/tests. 2016 is the year when SDS becomes widely used in production deployments. The technology has reached a good level of maturity, developed by early adopters and now mainstream companies are going to deploy it in order to get the benefits delivered by software-defined storage and distributed storage.

Shingled Magnetic Recording (SMR) Drives

SMR drives will be one of the storage trends of 2016. The economics of this new class of media is clear already – SMR HDDs are nearly half the cost of the closest regular HDD available. Other than that, the attraction of SMR drives is the ability to have an online archive at a very reasonable cost rather than an offline tape archive. Also, the software stacks and products incorporating them are just coming into place. They fit between tape and HDDs, at the same time displacing both to a degree. The tape will be dead as a backup technology of the mass market and SMR drives will make it really dead.

New storage tiers

The NVM and SMR drives will create new tiers of storage. NVM will fit between flash and RAM, displacing the highest end of the flash arrays, and enabling a small segment of new applications. SMR drives will fit between the tape and regular HDDs, displacing both to a degree.

Further Systems Specialization

At the medium scale (100 TB – 1 PB) there is an increasing realization that you probably need several different systems for the different use cases. The best architecture for an archival system is very different for the best architecture for primary storage. So in 2016 mid-sized storage users will look even more to solve different requirements with different systems.

Death of HDD-only and steep decline of cached-HDD systems

Enterprises finally realize that HDD systems, even when cached and centralized architectures don’t work for their virtualized workloads. We still see quotes by major vendors offering hard disk-based systems, where the customer clearly stated they want high performance. This will not fly anymore in 2016. This is all part of the specialization trend: different problems = different solutions.

The pursuit to reduce Complexity

Complexity is a major pain point, getting worse with all the latest technologies that companies have to keep up with and manage. Hyper-convergence promises to address these issues to a large degree, but only in the smaller storage deployments. Cloud operators also help a lot by hiding the complexity of their infrastructure behind an easy-to-use web interface.

On a larger scale – when you have to build a public or a private cloud – the sensible solution is to consider software-defined technologies. Most of them rely on standard hardware (and standard skillset), thus they reduce complexity as well as vendor lock-in. This promises to be one of the huge storage trends in 2016.

Pricing of storage systems changes

It is not about storage anymore – compute network and storage become one integral system. This has a considerable impact on how you compare solutions – it is not $/GB and $/IOPs of the storage system anymore. Rather companies have to compare functionality and the total cost of their alternative stacks before choosing.

New approach to Backup and Disaster Recovery (DR)

There is an increased interest for multi-site deployments. In the larger cloud deployments, the general approach seems to be multiple sites connected over the Internet, with an exposed choice to the customer where the cloud should keep backup copies of their data. We don’t see any interest in tape libraries. The modern approach to backup and archiving seems to be large HDDs with wide erasure coding. So backups will be either online or nearly online (within seconds of powering up the HDDs). We also see increased interest in being able to run applications from the backup system even with decreased performance. The line between backup and DR is blurring.

Storage trends and predictions – Summary

On the highest level, there is a Paradigm Shift from “storage as a separate subsystem” (i.e. standalone SAN box) to “software-defined storage”. Companies will grasp the interplay between applications, compute, network and storage. Storage is becoming an integral part of the entire stack and tightly integrated with network and compute (and applications running on top).

Performance requirements are rising and SSD-level of performance is now the standard. Use cases multiply which drives further specialization of storage systems and new tiers of storage.

With impact lower than the Paradigm Shift new technologies are adding functionality, but also complexity, which the end users are trying to manage while innovating to keep their competitive edge. These are the top storage trends and predictions we see for 2016.

We’ll be glad to have your views in the comments!