Jabil is working with Microsoft to look at how it could use Project Brainwave to quickly and accurately use AI to scan images and flag false positives – or items that aren’t actually defective. That more detailed analysis will free up the people who manually check for defects to focus on the more complex cases.

All those seconds saved add up, said Ryan Litvak, Jabil’s IT manager, as manufacturing customers are always eager to find any efficiency gains they can pass on to their customers.

“It’s highly competitive, so anything you can do to get an advantage for the customer is going to provide incremental improvement,” he said.

The Project Brainwave preview includes the ability for customers to do ultra-fast image recognition for applications such as the one Jabil is piloting, and it lets people do AI-based computations in real time, instead of batching it into smaller groups of separate computations. It works on TensorFlow, one of the most commonly used frameworks for doing AI calculations using deep neural networks, a method that is roughly modeled on theories about how the brain works. In addition, Microsoft is working on building the capability to support Microsoft Cognitive Toolkit, another popular framework for deep learning.

Microsoft also is announcing a limited preview to bring Project Brainwave to the edge, meaning customers could take advantage of that computing speed in their own businesses and facilities, even if their systems aren’t connected to a network or the Internet.

“We’re making real-time AI available to customers both on the cloud and on the edge,” said Doug Burger, a distinguished engineer at Microsoft who leads the group that has pioneered the idea of using FPGAs for AI work.

For Jabil, Litvak said the effort to make Project Brainwave available on the edge, which would allow his company to put it directly in manufacturing facilities, is key to making it practical and profitable to deploy across all its operations.

Jabil is one of the largest and most technologically advanced manufacturing solutions providers in the world. Their customers come to them at every stage of their business plans, from an idea on a napkin to a full-fledged product ready for assembly, but they all have one thing in common – the need to operate as efficiently and cost-effectively as possible. That means the ability to save even a few milliseconds of time, and a little bit of money, by not sending each calculation into the cloud adds up.

Litvak said the ability to bring Project Brainwave to the edge could allow Jabil to scale beyond the pilot and potentially bring the capability to all its manufacturing operations.

Jabil also is looking at ways to use AI on Project Brainwave to better predict when manufacturing operations need maintenance, as a way to cut downtime.

From research to product

The public preview of Project Brainwave comes about five years after Burger, a former academic who works in Microsoft’s research labs, first began talking about the idea of using FPGAs for more efficient computer processing. As he refined his idea, the current AI revolution kicked into full gear. That has created a massive need for systems that can process the large amounts of data required for AI systems to do things like scan documents and images for information, recognize speech and translate conversations.

Burger says Project Brainwave is perfect for the demands of AI computing. Project Brainwave’s hardware design can evolve rapidly and be remapped to the FPGA after each improvement, keeping pace with new discoveries and staying current with the requirements of the rapidly changing AI algorithms.

An FPGA also can be quickly reprogrammed to respond to new advances in AI, making it more flexible in such a fast-changing field than other types of computer chips.

The demand for systems that can handle AI workloads quickly and at reasonable cost is only expected to grow. That’s because companies are looking at more sophisticated uses of AI, such as to analyze unstructured data like videos, and they are developing more sophisticated AI algorithms that can do things like search those videos for all footage of cities near oceans.

“We’re going to need a lot more computing power,” Burger said.

From Bing and Azure to all businesses

FPGAs weren’t new when Burger and his team started exploring the idea that led to Project Brainwave, but until then no one had seriously considered them for large-scale computing. So the team set out to prove that they could be used in more practical scenarios.

Burger and his team found partners for their project, initially called Project Catapult, in the Bing search engine and Azure cloud teams.

Eric Chung, a senior researcher in Microsoft’s Silicon Systems Futures group and the technical lead on Project Brainwave, describes Bing as “constrained by low latency.” That’s a technical way of saying that when Bing customers type in a search query, they expect nearly instantaneous results.

That means that Bing engineers are constantly looking for ways to improve the quality of their results without giving up even a millisecond of speed – at a time when the amount and types of data search engines cull through is only growing.

Using FPGAs, the team was able to quickly incorporate deep neural network-based search technology into Bing, vastly speeding up the system’s ability to serve up results.

“That was only possible because it was done in programmable hardware,” said Steve Reinhardt, a partner hardware engineering manager and a leader of Bing’s hardware acceleration efforts.

Now, Ted Way, a senior program manager in Azure Machine Learning, said the programmable hardware has another big advantage – it will be easy to reprogram it as new innovations come along, whereas other systems may require you to update the hardware, which could take months or years.

Azure also has been using FPGAs for the past several years to accelerate Azure’s cloud network.