What if your pocket video camera had the intelligence to figure out that the sportscar you’re recording as it zips down the street is accelerating and not about to stop? What if it could sense that the person standing in front of the car might get run over if he doesn’t move? It’s not as far-fetched as it sounds, at least in the world of the Pentagon’s Defense Advanced Research Research Projects Agency (DARPA).

DARPA’s just-launched Mind’s Eye program aims to develop visual intelligence for unmanned systems. Once perfected, the agency anticipates that such systems could perform ground surveillance in military operations–a job currently relegated to Army scouts and Marine Corps Force Reconnaissance troops.

DARPA explains:

Humans perform a wide range of visual tasks with ease, something no current artificial intelligence can do in a robust way. They have inherently strong spatial judgment and are able to learn new spatiotemporal concepts directly from the visual experience. Humans visualize scenes and objects, as well as the actions involving those objects and possess a powerful ability to manipulate those imagined scenes mentally to solve problems. A machine-based implementation of such abilities is broadly applicable to a wide range of applications, including ground surveillance.

Just how difficult is it to create a visually intelligent machine? DARPA’s 12 research teams working on the initiative, which include Carnegie Mellon University, Co57 Systems, Colorado State University, Jet Propulsion Laboratory/CALTECH, and MIT, are responsible for creating a software system for unmanned ground vehicles that uses existing computer vision and artificial intelligence technology and makes “novel contributions in visual event learning, new spatiotemporal representations, machine-generated envisionment, visual inspection and grounding of visual concepts.”

There is, in other words, a lot of work to do before Mind’s Eye turns into a real product. Once it does, however, the real-world applications are endless–replacing reporters with machines in war-torn areas, better home security systems, and improved business surveillance systems (no more security guards in the surveillance room) are just some of the developments we could see in the coming years.

Ariel Schwartz can be reached on Twitter or by email.