TORONTO — Big data applications have already driven the need for architectures that put memory closer to compute resources, but artificial intelligence (AI) and machine learning are further demonstrating how hardware and hardware architectures play a critical role in successful deployments. A key question, however, is where the memory is going to reside.

Research commissioned by Micron Technology found that 89% of respondents say it is important or critical that compute and memory are architecturally close together. The survey, carried out by Forrester Research, also found that memory and storage are the most commonly cited concerns regarding hardware constraints limiting AI and machine learning today. More than 75% of respondents recognize a need to upgrade or re-architect their memory and storage to limit architectural constraints.

AI compounds the challenges already unearthed by big data and analytics requirements because machine learning does multiple accumulation operations on a vast matrix of data over neural networks. These operations are repeated over and over as more results come in to produce an algorithm that is the best path and the best choice each time — it learns from working on the data.

Because there’s so much data, said Colm Lysaght, Micron’s vice president of corporate strategy, a common solution for getting the necessary working memory is simply to add more DRAM. This is shifting the performance bottleneck from raw compute to where the data is. “Memory and storage is where the data is living,” he said. “We have to get it to a CPU and then back again, over and over again, as these vast data sets are being worked on.”

Finding ways to bring compute and memory closer together means saving power because data isn’t being shuttled around as much, Lysaght said. “It’s increasing performance because more things can happen right where they need to happen.”

Micron sees existing memory and storage technologies such as DRAM and 3D NAND SSDs providing the hardware for AI architectures but is also researching newer technologies, such as processor-on-memory architectures, while supporting pioneering startups.

There is a number of different approaches to creating better architectures, Lysaght said. One example is neuromorphic processors that use a neural network internally and break up the internal number of cores into a larger number of smaller cores. “Because there is a large matrix of data that’s being worked on, having many more cores doing relatively simple operations over and over again is a better solution,”Lysaght said.

One memory company that’s interested in developing new architectures is Crossbar Inc. Along with Gyrfalcon Technology Inc., mtes Neural Networks Co. (mtesNN), and RoboSensing Inc., it formed SCAiLE (SCalable AI for Learning at the Edge), an AI consortium dedicated to delivering an accelerated, power-saving AI platform.

Sylvain Dubois, vice president of strategic marketing and business development at Crossbar, said that the group will combine advanced acceleration hardware, resistive RAM (ReRAM), and optimized neural networks to create ready-made, power-efficient solutions with unsupervised learning and event-recognition capability.

Dubois said that the challenge for many companies is that they want AI on a device but have no idea how to do it, whether it’s a smart speaker, smart camera, or smart TV. The goal of the consortium is to provide a platform that brings all of the necessary pieces together.

Crossbar’s contribution is around the memory — specifically ReRAM — that will help process the data that comes into a machine-learning system via a wide range of inputs, including text, keywords, GPS coordinates, and visual data from sensors — all of it unstructured.

Dubois envisions a memory array architected to be read by specific processing codes for each of these instances in a very wide and highly parallel way, where a thousand beats are read in parallel in an edge device.

“If you’re a match, then you know at the edge what to do,” Dubois said. “But if you don’t have a match, then this is what we call the learning curvature.”

For example, in the case of a camera sensor, he said, the system will be able to store new events or a set of features in a spare location of the ReRAM array. “The next time you have a similar event passing in front of this camera, the camera itself will be able to detect it without any training in the cloud,” Dubois said.