Exactly one year ago, DARPA announced a characteristically scifi-inspired mission: to create a search engine that could find things on the deep web that Google's crawlers would miss. The so-called Memex project is now well underway, and for the first time we're getting a look at the crime-fighting search engine in action.


Pardon the clichéd Philip K. Dick reference, but Memex looks a bit like something you'd see in Minority Report. The Pentagon's research agency offered Scientific American and 60 Minutes exclusive looks at the technology, and the features sound absolutely mind-bending.

On the surface level, Memex works like a search engine that spreads its tentacles into the deep web and darknet. Since the likes of Google and Bing only index about 10 percent of the web, this basic functionality is key to tracking criminal movements, especially those of human traffickers who maintain the lowest of profiles.


So take this scenario: If a criminal investigator has the link to an ad posted by a human trafficker, the investigator would have a hard time finding other clues since human traffickers pull these ads before Google has a chance to index them. Memex circumvents that.

The next generation technology queries a much broader swatch of the internet, including deep web and darknet links, to find connections between the search term and its results, not to mention connections between the results.

After the initial search, Memex will then produce a so-called "data wake" that shows all of the other pages that are related to the links that you clicked on but might not otherwise see. It looks like this:


That's just one trick. The super-charged web crawlers are also able to track the movements of human traffickers by watching where they post ads. Again, these are links that would slip through Google's tracks. The movements can them be compiled into heat maps or maps of a criminal's movements:


Things get really futuristic when you take into account the latest Memex feature. The technology is currently being beta tested by two district attorneys' offices, a law enforcement agency, and a nongovernmental organization. The next stage of testing, due to start with a broader group of beta testers in a few weeks, is where things start to sound more like Minority Report. From Scientific American:

One of the main objectives of this round is to test new image search capabilities that can analyze photos even when portions that might aid investigators—including traffickers' faces or a television screen in the background—are obfuscated. Another goal is to try out different user interfaces and to experiment with streaming architectures that assess time-sensitive data.


You heard that right. DARPA wants to catch criminals by looking at reflections in TV screens, the same way that Tom Cruise figures out the details of crimes before they happen. Of course, Memex can't see into the future. However, the idea is that the technology will stop human traffickers before they hurt more people. All DARPA need now are those weird gloves, a room-sized screen, and some mutated, prophetic humans soaking in a special chamber, and the agency will finally be a full-on Philip K. Dick vision come true. [SciAm, 60 Minutes]