Exactly one year ago, DARPA announced a characteristically sci-fi-inspired mission: to create a search engine that could find things on the deep web that Google's crawlers would miss. The so-called Memex project is now well underway, and for the first time we're getting a look at the crime-fighting search engine in action.
Pardon the clichéd Philip K. Dick reference, but Memex looks a bit like something you'd see in Minority Report. The Pentagon's research agency offered Scientific American < and news investigation show 60 Minutes exclusive looks at the technology, and the features sound absolutely mind-bending.
On the surface, Memex works like a search engine that spreads its tentacles into the deep web and dark net. Since the likes of Google and Bing only index about 10 per cent of the web, this basic functionality is key to tracking criminal movements, especially those of human traffickers who maintain the lowest of profiles.
So take this scenario: if a criminal investigator has the link to an ad posted by a human trafficker, the investigator would have a hard time finding other clues since human traffickers pull these ads before Google has a chance to index them. Memex circumvents that.
The next generation technology queries a much broader swathe of the internet, including deep web and dark net links, to find connections between the search term and its results, not to mention connections between the results.
After the initial search, Memex will then produce a so-called "data wake" that shows all of the other pages that are related to the links that you clicked on but might not otherwise see. It looks like this:
That's just one trick. The super-charged web crawlers are also able to track the movements of human traffickers by watching where they post ads. Again, these are links that would slip through Google's tracks. The movements can them be compiled into heat maps or maps of a criminal's movements:
One of the main objectives of this round is to test new image search capabilities that can analyze photos even when portions that might aid investigators — including traffickers' faces or a television screen in the background — are obfuscated. Another goal is to try out different user interfaces and to experiment with streaming architectures that assess time-sensitive data.
You heard that right. DARPA wants to catch criminals by looking at reflections in TV screens, the same way that Tom Cruise figures out the details of crimes before they happen. Of course, Memex can't see into the future. However, the idea is that the technology will stop human traffickers before they hurt more people. All DARPA need now are those weird gloves, a room-sized screen, and some mutated, prophetic humans soaking in a special chamber, and the agency will finally be a full-on Philip K. Dick vision come true.
[SciAm, 60 Minutes]