Nvidia and IBM have developed an interconnect that will be integrated into future graphics processing units, letting GPUs and CPUs share data five times faster than they can now, Nvidia announced today. The fatter pipe will let data flow between the CPU and GPU at rates higher than 80GB per second, compared to 16GB per second today.

NVLink, the interconnect, will be part of the newly announced Pascal GPU architecture on track for release in 2016.

GPUs have become increasingly common in supercomputing, serving as accelerators or "co-processors" to help CPUs get work done faster. In the most recent list of the world's fastest 500 supercomputers, 53 systems used co-processors and 38 of these used Nvidia chips. The second and sixth most powerful supercomputers used Nvidia chips alongside CPUs. Intel still dominates, providing processors for 82.4 percent of Top 500 systems.

"Today's GPUs are connected to x86-based CPUs through the PCI Express (PCIe) interface, which limits the GPU's ability to access the CPU memory system and is four- to five-times slower than typical CPU memory systems," Nvidia said. "PCIe is an even greater bottleneck between the GPU and IBM Power CPUs, which have more bandwidth than x86 CPUs. As the NVLink interface will match the bandwidth of typical CPU memory systems, it will enable GPUs to access CPU memory at its full bandwidth... Although future Nvidia GPUs will continue to support PCIe, NVLink technology will be used for connecting GPUs to NVLink-enabled CPUs as well as providing high-bandwidth connections directly between multiple GPUs."

Nvidia further explained that "GPUs have fast but small memories, and CPUs have large but slow memories—accelerated computing applications typically move data from the network or disk storage to CPU memory and then copy the data to GPU memory before it can be crunched by the GPU. With NVLink, the data moves between the CPU memory and GPU memory at much faster speeds, making GPU-accelerated applications run much faster."

High-performance computing, data analytics, and machine learning will benefit from the new architecture. Nvidia also said the NVLink interconnect presents a path to "highly energy-efficient and scalable exascale supercomputers," but that milestone is still probably at least a few years away.

Separately from NVLink, a feature in Pascal, called "Unified Memory," will simplify programming by letting coders treat the CPU and GPU memory as a single block. Pascal's design also "stacks DRAM chips into dense modules with wide interfaces and brings them inside the same package as the GPU," Nvidia said. "This lets GPUs get data from memory more quickly—boosting throughput and efficiency." A module that houses Pascal and NVLink is one-third the size of standard boards, which means "they’ll put the power of GPUs into more compact form factors than ever before."