What is Edge AI, and why does it matter?

Traditionally, AI solutions have needed a large amount of parallel computational processing power. So for a long time, a requirement of AI-based service was server-based Internet connectivity. But solutions that require real-time action need on-device computation—this is where edge AI enters the picture.

You can use GPU-based devices, but it makes the process costly, and with this come the problems of bloated size and high energy consumption. But more and more, edge AI is becoming an essential part of the ongoing deep learning revolution, both in terms of research and innovation.

With ever-more powerful handheld devices in the hands and pockets of billions around the world, we’re steadily seeing increased demand for on-device AI computations. As such, the development of mobile processors has become more AI-focused, with dedicated hardware for machine learning.

Quick overview of current hardware (Intel, etc)

USB Accelerator

Though this is the first Edge TPU, we’ve seen a couple of similar AI-dedicated hardware before, such as:

Google Vision Kit and Intel® Neural Compute Stick

Coral Beta

The TPU—or Tensor Processing Unit—is mainly used by Google data centers. For general users, it’s available on the Google Cloud Platform (GCP), and to try it free you can use Google Colab.

Cloud TPU v3

Google first showcased their Edge TPU in a demo at CES 2019 (and again at this year’s TensorFlow Dev Summit). And in March, they released Coral Beta.

The Beta release consists of a Dev Board and USB Accelerator, and previews of a PCI-E Accelerator and System-on-Module (SOM) for production purposes.

What can you do with the Edge TPU?

With the Edge TPU, you can train models on-device, but for now, it only supports classification models to be retrained on the device using transfer-learning, based on the weight imprinting technique proposed in Low-Shot Learning with Imprinted Weights. This technique opens up many possibilities for real-time systems. Furthermore, it’s the fastest inference device of its kind.