The government agency that birthed the Internet is developing a sophisticated search engine for video, and when complete will allow intelligence analysts to sift through live footage from spy drones, as well as thousands of hours worth of archived recordings, in order to spot a variety of selected events or behaviors. In the past month, the Defense Advanced Research Projects Agency announced nearly $20 million in total contracts for private firms to begin developing the system, which is slated to take until at least 2011 to complete.

According to a prospectus written in March but released only this month, the Video and Image Retrieval and Analysis Tool (VIRAT) will enable intel analysts to "rapidly find video content of interest from archives and provide alerts to the analyst of events of interest during live operations," taking both conventional video and footage from infrared scanners as input. The VIRAT project is an effort to cope with a growing data glut that has taxed intelligence resources because of the need to have trained human personnel perform time- and labor-intensive review of recorded video.



Diagram of the VIRAT system concept

The DARPA overview emphasizes that VIRAT will not be designed with "face recognition, gait recognition, human identification, or any form of biometrics" in mind. Rather, the system will search for classes of activities or events. A suggested partial list in the prospectus includes digging, loitering, exploding, shooting, smoking, following, shaking hand, excahnging objects, crawling under a car, breaking a window, and evading a checkpoint. As new sample clips are fed into the system, it will need to recognize the signature features of new classes of search terms.





How VIRAT would work in action

VIRAT will be rolled out in three phases, each assigned to a different contractor in order to prevent conflicts of interest. The first phase, design of an initial prototype, will be handled by New York–based Kitware, Inc., which The Washington Post reports is heading a consortium of 9 companies and universities that will work on the $6.7 million project. (Correction: The Post initially misidentified the consortium as comprising 19 members.) Later phases will refine and optimize the search algorithm, then demonstrate its capabilities on real-world data. Massachussets-based BAE Systems National Security Solutions and Lockheed Martin were also awarded VIRAT contracts, presumably for these later phases, totalling $7.2 million and $5.5 million respectively.

By the end of the final phase of development, according to DARPA's plan, VIRAT will process at least 58 megapixels per second, with an accuracy rate of 95 percent.