Home » Featured » DARPA’s Creating Search Engine to Crawl the Deep Web
Click Here To Hide Tor

DARPA’s Creating Search Engine to Crawl the Deep Web

As it was mentioned in cbsnews (video available here) Memex, a powerful new search tool that goes beyond the realm of Google, Yahoo, and Bing, is launched by DARPA:
This powerful new search engine developed by DARPA, the U.S. military’s Defense Advanced Research Projects Agency and was announced last year. The inventor of Memex, Chris White:

“The internet is much, much bigger than people think,” White said. “By some estimates Google, Microsoft Bing, and Yahoo only give us access to around 5% of the content on the Web.” That leaves a lot of room for bad actors to operate freely in the shadows.

White says that Memex goes far beyond the realm of traditional search engines and gives law enforcement a powerful new tool to search the “dark web,” where criminals buy, sell, and advertise in the illegal weapons trade and sex trafficking.

“The easiest way to think about Memex is: How can I make the unseen seen?” said Dan Kaufman, director of the information innovation office at DARPA.

“Most people on the internet are doing benign and good things,” Kaufman said. “But there are parasites that live on there, and we take away their ability to use the internet against us– and make the world a better place.”

According to published reports, including one from Carnegie Mellon University, the NYDA’s Office is one of several law enforcement agencies that have used early versions of Memex software over the past year to find and prosecute human traffickers, who coerce or abduct people—typically women and children—for the purposes of exploitation, sexual or otherwise. “Memex”—a combination of the words “memory” and “index” first coined in a 1945 article for The Atlantic—currently includes eight open-source, browser-based search, analysis and data-visualization programs as well as back-end server software that perform complex computations and data analysis.

DARPA has asked researchers to develop advanced web-crawler software to reach sites and resources that have sophisticated crawler defenses. Memex operators would then be able to access the indexed domain-relevant content with much greater precision and ease than is currently possible.

Memex, DARPA says, will be first employed against human trafficking, which, “especially for the commercial sex trade, is a line of business with significant Web presence to attract customers and is relevant to many types of military, law enforcement, and intelligence investigations.”

DARPA says that dark places online where trafficking occurs enables “a growing industry of modern slavery” that can be stopped with Memex capabilities.

“An index curated for the counter trafficking domain, including labor and sex trafficking, along with configurable interfaces for search and analysis will enable a new opportunity for military, law enforcement, legal, and intelligence actions to be taken against trafficking enterprises,” DARPA’s solicitation announcement reads.

How DARPA would catch traffickers without “deanonymizing” someone, though, the agency does not explain. Nor does it address just how far it wants to out anyone hiding in the deep web for legitimate reasons, whether they are journalists, whistleblowers, activists, and the like.

SourcesGizmodo, scientificamerican , RTCbsnews

 

4 comments

  1. This just means that hidden servers, in addition to picking their Guard relay very, very carefully, also need to be mobile, moving from one region to another periodically, perhaps even over a wireless connection. Such will mean that systems, such Memex, will need to “start over” in their attempts to deanonymize a hidden server. Eventually, with enough (additional) hopes, it will simply be uneconomical for TLAs/LE to target hidden services, forcing them to stop.

    • @other ANON: this is to track websites and servers hosting darknet sites, not tracking users over darknet. And in neither case will it ever become “uneconomical”, they don’t over tax and print money so they can decide who to chase after and who not to…they do it so they can take it to the ends of the universe if need be.

      • Anarchist-Capitalist

        Different Anon here.

        No, I’m an Anarcho-Capitalist.

        The Government doesn’t have the power to take anything to the ends of the universe. The Government has no power to wage war anymore. It’s gone. Government is decaying and we are witnessing it now. It will catch on soon, and Government will dissolve overnight.

  2. I just signed up with nordsVPN. They are fast on communication and hooking up for Linux is fantastic. In Short, go with nordsVPN

Leave a Reply

Your email address will not be published. Required fields are marked *

*

Captcha: *