PARIS — In the fast-growing markets for factory automation, IoT, and autonomous vehicles, CMOS image sensors appear destined for a role capturing data not for human consumption but for machines to see what they need to make sense of the world.

CMOS image sensors “are becoming more about sensing rather than imaging,” said Pierre Cambou, activity leader, MEMS & Imaging at Yole Développement. The Lyon, France-based market research and technology analysis company boldly predicts that by 2030, 50% of CMOS image sensors will serve “sensing” devices.

Luca Verre

Paris-based Prophesee SA (formerly known as Chronocam) styles itself as a frontrunner in that revolution. A designer of advanced neuromorphic vision systems, it advocates an event-based approach to sensing and processing. Prophesee’s bio-inspired vision technology has been deemed too radically different from conventional machine vision — and perilously “ahead of its time.” But Luca Verre, co-founder and CEO of Prophesee, told us that this is no longer the case.

In a one-on-one interview here, Verre said that his company has secured its Series B plus funding (the startup raised $40 million in funding in the last three years). It now has a partnership deal with a large unnamed consumer electronics company. Most importantly, Prophesee is now advancing its neuromorphic vision system from the usual technology concept pitch to promoting its reference system for tinkering by developers.

Prophesee’s first reference design, available in VGA resolution, consists of Prophesee’s Asynchronous Time-Based Image Sensor (ATIS) chip and software algorithms. The ASIC will be manufactured by a foundry partner in Israel, said Verre — most likely Tower Jazz.

The company declined to detail its ASIC and the specification of the reference design. Prophesee said that it is planning on a formal product announcement in several weeks.

Nonetheless, the startup reached a milestone when the reference design proved able to offer system designers the opportunity to see and experience just what an ATIS can accomplish in data sensing. The ATIS will be characterized by its high temporal resolution, low data rate, high dynamic range, and low power consumption, said Prophesee.

Cameras are bottlenecks

Makers of cameras for machine-vision systems — whether in smart factories, IoT, or autonomous vehicles — have begun to heed the event-based approach promoted by Prophesee’s co-founders such as Ryad Benosman and Christoph Posch.

With all of the detailed visual information that traditional cameras can capture, “the camera has become a technology bottleneck,” said Verre. Unquestionably, cameras are the most powerful sensing device. Yet for visual data in automation systems, surveillance cameras, or highly automated vehicles, cameras could slow down the processing.

Consider self-driving cars, said Verre. The central processing system inside the vehicle is bombarded with data from cameras, lidars, radars, and other sources. The key to manage this overload is figuring out how best to “reduce the amount of raw data” streamed from sensors. The sensors should only capture data that matters to “a region of interest,” said Verre.

As Prophesee explained in past interviews with EE Times, the company’s event-driven vision sensors are inspired by biology. This perception derives from the co-founders’ research on how the human eye and brain work.

Ryad Benosman, Prophesee’s co-founder, told us that human eyes and brains “do not record the visual information based on a series of frames.” Biology is much more sophisticated. “Humans capture the stuff of interest — spatial and temporal changes — and send that information to the brain very efficiently,” he said. That’s principally what Prophesee’s ATIS does.

Noting that the ATIS is not bound by frames, Verre explained, “Our technology will not have to miss important events that might have happened between frames.”

[Sponsored: Bernecker & Rainer (B&R) ACOPOS P3 – Teardown]

In short, what Prophee’s ATIS offers is everything that frame-based image sensors is not. In the view of another co-founder, Christoph Posch, “Frame-based methodology results in redundancy in the recorded data, which triggers higher power consumption.” He said, “Results include inefficient data rates and inflated storage volume. Frame-based video, at 30 or 60 frames per second, or even a much higher rate, causes a catastrophe in image capturing.”

Event-driven approach for lidars

Verre last week disclosed to us that Prophesee is exploring the possibility that its event-driven approach can apply to other sensors such as lidars and radars. Verre asked: “What if we can steer lidars to capture data focused on only what’s relevant and just the region of interest?” If it can be done, it will not only speed up data acquisition but also reduce the data volume that needs processing.

Phrophesee is currently “evaluating” the idea, said Luca, cautioning that it will take “some months” before the company can reach that conclusion. But he added, “We’re quite confident that we can pull it off.”

Asked about Prophesee’s new idea — to extend the event-driven approach to other sensors — Yole Développement’s analyst Cambou told us, “Merging the advantages of an event-based camera with a lidar (which offers the “Z” information) is extremely interesting.”

Noting that problems with traditional lidars are tied to limited resolution — “relatively less than typical high-end industrial cameras” — and the speed of analysis, Cambou said that the event-driven approach can help improve lidars, “especially for fast and close-by events, such as a pedestrian appearing in front of an autonomous car.”

The downside is that lidar hardware would have to be changed, he added. More importantly, though, Prophesee needs a strong buy-in from lidar companies to this event-driven approach.

Cambou said, “Sure, this is always the problem for a technology startup.” He pointed out that Mobileye needed some lead customers such as Volvo and a Tesla [before having its technology going mainstream and getting broadly accepted]. Movidius, now an Intel company, needed DJI [to become successful]. “Prophesee will need a strong partner in order to have its solution largely adopted,” said Cambou.

“Given the market drivers in the realm of robotic vehicles (safety first, technology-driven, not so cost-conscious),” he added, “This should be possible.”

Although Cambou expressed his concerns about a large player such as Google depending on a small startup for its technology, he brushed off this concern by noting that the small volume involved makes this less of an issue.