A computational graph of a deep learning model, visualized by Graphcore’s IPU compiler, Poplar. Millions of vertices and edges represent computation processes and communication between processes, respectively. Clusters represent intensive communication between processes in each layer of the network, with lighter communication between layers. (Source: Graphcore)

GPUs are widely used to accelerate AI computing, but are the limitations of GPU technology slowing down innovation in the development of neural networks?

In a recent interview with EETimes (Graphcore CEO Touts ‘Most Complex Processor’ Ever), Nigel Toon, CEO of Graphcore, explained that while GPUs are good at running convolutional neural networks (CNNs), they are not suitable for running the more complex types of neural network needed for reinforcement learning and other futuristic techniques.

“A GPU is a pretty good solution — if all you’re doing is basic, feed-forward CNNs. The problem comes when you start to have more complex neural networks. Rather than just doing it a layer at a time, I want to be able to go through some layers then feed back, and I want to be able to store information on the side which I can use as context information as I look at the next data. And if my data is changing — so, rather than it being millions of static images that I can feed in in parallel — if it's video and I'm interested in sequential frames, it's much harder to feed that in in parallel and take advantage of the wide SIMD paths in a GPU,” he said.

Visionary Approval

As part of our longer conversation, Toon noted that Graphcore has captured the interest of such AI visionaries as Demis Hassabis, a founder of DeepMind, and the founders of OpenAI, including Ilya Sutskever, along with many other leading researchers in machine learning. Graphcore worked with these researchers to design the IPU architecture based on the kinds of problems that they want to solve.

Recommended

Graphcore CEO Touts ‘Most Complex Processor’ Ever

“All the innovators we spoke to said [using GPUs] is holding them back from new innovations,” he said. “If you look at the types of models that people are working on, they are primarily working on forms of convolutional neural networks because recurrent neural networks and other kinds of structures, [such as] reinforcement learning, don't map well to GPUs. Areas of research are being held back because there isn't a good enough hardware platform, and that's why we're trying to bring [IPUs] to market.”

Processor Development

Toon points to the development of ASIC-type accelerators that are built to accelerate specific neural networks, as well as increased interest in FPGA solutions for AI, as proof that GPUs can’t do the job well enough. There is a need, he says, for an easy-to-use processor that is designed from the ground up, specifically for machine intelligence.

“What you need to do is to extract parallelism in many different dimensions. So, rather than having an SIMD processor, what we need is a multiple-instruction, multiple-data machine,” he said. “We need to solve the problems of: ‘How can we access the memory in real time, during the compute?’ ‘How can I take pieces of data from here and there, gather that together, do the compute, and then scatter the answer back somewhere else?’ These are all the things that we have been solving with the IPU processor.”

Graphcore’s IPU is designed to hold an entire machine learning model, providing massive parallelism, plenty of memory, and crucially, tons of memory bandwidth. The result is very fast neural networks. Compared to GPUs, IPUs are up to 5x faster for CNNs, such as those used in static image processing today, but they are up to 50x faster for more complex models.

Neural network complexity increases as the industry develops techniques, such as reinforcement learning, in which the data is fed back through the network more than once to try and help the system learn from its experience. This is important in situations where we want to understand context, such as spoken conversations. These complex neural networks are undoubtedly the future of AI, but how quickly they evolve depends on the development of specialized AI accelerators — like Graphcore’s.

Read the full interview with Nigel Toon here: Graphcore CEO Touts ‘Most Complex Processor’ Ever.