Skip to main content
  1. Home
  2. Web
  3. Guides

Search like a spy with these tips from a declassified NSA document

Drew Prindle
By
declassified NSA document

In case you weren’t already aware, the Internet is home to an unfathomably large amount of information. Seriously – there’s probably at least 100 terabytes taken up just by funny cat GIFs, let alone all the meaningful, educational, enlightening stuff. And with so much information online, even government agencies sometimes need help making sense of it all. 

Back in 2007, to help their field agents use the web more effectively, the NSA commissioned a guide to be written, and thanks to a recent FOIA request from MuckRock, the 643 page document was recently released to the public. The guide, entitled Untangling the Web: a Guide to Internet Research, is absolutely stuffed full of great information – or at least it was great information. 

Related Videos

The NSA had some pretty slick tricks up their sleeve back in 2007

The book stands as a testament to how quickly the web changes. It was written just six years ago, but has suffered a large degree of technological attrition – many of the websites, tools, and services recommended in it no longer exist. 

Related

That being said, however, there are still a few things in the book that are worth reading. Taken from a broad standpoint, many of the methods outlined here are still useful – it’s just that the tools used to execute them have changed. The NSA had some pretty slick tricks up their sleeve back in 2007, so I’ve taken the liberty of reproducing them for all you aspiring international super spies out there. Here’s the highlights:

Google Hacking

One of the most dastardly chapters in the manual is the section on “Google Hacking,” which the authors describe as “using publicly available search engines to access publicly available information that almost certainly was not intended for public distribution.” Most of the manual is pretty out of date, but this section is just as relevant and useful as it’s ever been. Here’s what the book recommends:

The first part of a good Google hack is knowing how to use Google’s search operators. These are nifty little words and symbols you can append to your queries to get more specific results. Google lists a few of them on their support page, but there are hundreds more that they don’t bother mentioning. It’s worth noting that you don’t need to memorize all of them since the same results can be achieved by using Google’s advanced search options, but that’s like using training wheels to ride a motorcycle. Badass international secret agent hackers don’t use training wheels. In order to be a true James Bond ass motherfucker, you should memorize a couple of these. 

The most useful one for devious spy-type activity is undoubtedly the filetype: operator. Using this, you can designate which type of files Google brings up. Here’s a quick briefing on some of the most common ones and what they’ll help you find:

  • filetype:xls will return a list of spreadsheets. These often contain personnel data, computer records, and financial information
  • filetype:doc or docx is good for internal working documents, reports, etc.
  • filetype:pdf is good for large documents of all types, and is widely used in academia, govnerment, and business 
  • filetype:ppt is good for retrieving briefings, which often contain company or government plans for the future

To maximize the effectiveness of these filetype searches and really start to dig up some dirt, the NSA recommends pairing them with boilerplate keywords. Try using terms like internal, budget, not for distribution, confidential, or company proprietary alongside your searches to pick up stuff that was unintentionally posted online. For example, if you’re looking for, say, classified NSA documents that might’ve been leaked on the web, try filetype:pdf site:nsa.gov “classified.”

Google SearchingAnother operator that might come in handy during some good ol’ fashioned espionage is the domain: operator. If used in conjunction with the right top level domain, you can use this operator to restrict results to webpages and documents hosted in specific countries. Let’s say you’re looking for spreadsheets full of passwords to the Russian Ministry of Defense. To point Google in the right direction, try searching filetype:xls domain:ru “password.”

Truth be told, these kinds of hacks were much more effective back in 2007, and nowadays companies and government organizations are pretty good about keeping internal documents off the Web. However, if you apply them in clever ways, these methods can still dig up a few goodies you probably weren’t meant to find.

Finding People

Untangling The Web has a pretty lengthy section on finding people, and despite the fact that it was written before rise of social networking, it’s still got a good list of tips for finding information on people. That being said, some of the suggestions are more relevant than others, so here’s the abridged version:

  1. Start by searching by name, address, email address, phone number (any personally identifiable information you have, really) on search engines like Google and Yahoo. This is kind of a no brainer, but it’s always a good place to start.
  2. If you know the person’s profession, you might find additional info on them in a database that contains stuff like licensing information. The US is really good about licensing people for all kinds of professions. Try other countries for similar information too.
  3. Property ownership and transactions are carefully recorded in the US and many such records are publicly available. This may also be true in other countries. Look for public databases of these records and transactions.
  4. If you know where the person works, that organization (be it government, academic, or corporate) might have a publicly accessible directory you can use to look them up
  5. Whois databases contain information about thousands of people associated with the Internet. If the person you’re looking for has a website, there’s a good chance their info can be found with a Whois lookup. The Whois databases maintained by ARIN, APNIC, AfriNIC, LACNIC, and RIPE are all searchable by name using their advanced search forms.

The authors then go on to mention a boatload of people finder sites, but pay them no heed. They’re all phooey. That was 2007, and we’ve got way better tools at our disposal these days. If you’re looking for a specific person, try searching their name, email address, or phone number on a site like Pipl, 123people, or Spokeo. These sites act as meta search engines, and gather data from a number of public records databases, social meidia profiles, and deep web resources (more on those in a minute).

Geolocating IP addresses

Let’s say you’ve got a name a few possible email addresses, but you can’t seem to track down the location of the high-profile narco-terrorist you’ve been assigned to take out. Not to worry – if you can manage to get your hands on his IP address, then hunting him down will be a piece of cake. Geolocating someone’s IP address is child’s play, and while it won’t give you their exact coordinates on a map, it’s a great tool for figuring a person’s approximate location on the globe. Back in 2007, IP geolocation tools were harder to find, but today they’re a dime a dozen. Just search Google for “IP geolocation” and click around until you find one that suits you. Personally, I prefer InfoSniper simply because it’s got a badass-sounding name and a nice visual interface.

data center

Searching the Deep Web

Google hacking is one thing, but if you can’t seem to find what you’re looking for on the Surface Web, chances are you’ll need to delve into the Deep Web. Also known as the Darknet, the Invisible Web, and similar variations, the Deep Web is basically anything that isn’t indexed by traditional web crawlers. To use Mike Bergman’s explanation, “searching on the Internet today can be compared to dragging a net across the surface of the ocean: a great deal may be caught in the net, but there is a wealth of information that is deep and therefore missed.”

The Deep Web is home to petabytes of information you can’t find on the surface, so you’re far more likely to dig up the dirt on somebody through the Deep Web – the hard part is just knowing where to look. The NSA lays out a short list of Deep Web resources to get you started – the only problem is that nearly all of them are no longer in operation. So, in their wake, we suggest using the following deep web resources:

  • CompletePlanet. This site bills itself as “The front door of the Deep Web,” and since it indexes over 70,000 different deep web databases, it’s definitely one of the best tools you have at your disposal
  • DeepWebTech offers a set of specialized search engines and browser plugins that crawl deep web databases. The search engines cover science, medicine, and business.
  • Scirus is a science-focused deep web portal that pulls information from a vast array of journals, periodicals, e-books, and other resources not traditionally indexed by search engines.
  • Infomine, one of the few resources listed in Untangling the Web that’s still up and running, is a fantastic resource for finding scholarly/academic information online.

Covering your Tracks

If you’re plugged in and in the midst of some serious webspionage, the last thing you want is to inadvertently leave traces of your activity. The NSA recommends a few methods for keeping your information secure, but oddly enough doesn’t go into great detail on the subject. The authors suggest things like using anti-spyware software, encrypting communications, and using strong passwords – pretty basic stuff. Not to worry though. We’ve put together an excellent introduction to staying anonymous online, which includes a host of programs and services that’ll keep your information hidden from prying eyes

Editors' Recommendations

The NSA might know what you look like, collects millions of digital images per day
Facial recognition

The National Security Agency (NSA) has ramped up its collection of digital facial images in the past four years, according to The New York Times, which cites documents from 2011 leaked by Edward Snowden. The intelligence agency is using software to glean “millions of images per day,” hoping to plug into the fire hose of photos included in emails, text messages and social media outlets, among other channels of digital communication.
Of those images, the NSA collects about 55,000 “facial recognition quality images,” which it says will add to its repository of identifying information (e.g., fingerprints) to track down suspected terrorists and other “targets of interest.”
An agency spokeswoman said the NSA didn’t have access to photos residing in state databases of driver’s licenses or passport photos of Americans, though she declined to say whether it had access to State Department photos of foreign visa applicants. She also declined to say whether the NSA collected images of Americans from Facebook and other social media.
There’s no telling how many people around the world have had images of their faces harvested by the NSA, according to the Times report. It’s also worth noting that neither federal privacy laws nor surveillance laws provide protections for facial images.
The NSA isn’t alone in its gathering of facial images. Back in April, the FBI released documents revealing it was building a biometric database that includes facial recognition data and may have records on up to a third of the U.S. population. Meanwhile, state and law enforcement agencies are leveraging driver’s licenses and Facebook for facial imagery (not to mention special mobile cameras), and the State Department may have the biggest facial imagery database in the federal government.
Not to be left out, the Department of Homeland Security is piloting projects at police departments to match suspects with faces in a crowd. But what makes the NSA’s endeavor unique, according to the Times report, is its ability to link facial images with private communications.
Joseph Atick, considered a pioneer in modern face recognition, recently expressed concern that the technology could open the door to mass surveillance. He said this application of face-matching technology would rob everyone of their anonymity and restrict people's normal behavior when they leave their homes.
[Image courtesy of rangizzz/Shutterstock]

Read more
It looks like Google has helped the NSA, but that could be a good thing
google gender and ethnic demographics statistics

Last year, a leaked top-secret document revealed the NSA's data collection program, called PRISM. Since then, tech companies have attempted to distance themselves from the NSA's activities, with Google, specifically, saying the only reason it cooperated with the NSA was because it was compelled to do so through secret court orders. However, email documents obtained by Al Jazeera show that there have been some backdoor discussions between the NSA and Google.
The documents show that NSA director General Keith Alexander invited Google CEO Schmidt to attend a "classified threat briefing" in order to discuss security issues. The email also revealed that, while Google, Apple, Microsoft, and other companies set out to define core security principles, other tech firms such as Dell, HP, and Intel set out to work with the NSA to minimize security threats to enterprise platforms.
While the emails do reveal that there was dialogue between the two firms that Google never revealed, keep in mind that companies are legally prohibited from revealing details about what the NSA asks of them. In any case, the emails only confirm that not only is Google not alone in maintaining a dialogue with the U.S. government, but that, given the size of the company, it's almost a given that such an occurrence would happen. There are likely a lot of ways Google could help the NSA out when it comes to security. While there is concern as to how the NSA uses information it receives from Google and other companies, the documents don't really implicate Google in any wrongdoing.

Read more
Apple, Google, and others are defying gov’ts to tell us when we’re being spied on

Tech companies such as Apple, Google, Facebook, Yahoo, and Microsoft reportedly plan to inform more people about government and law enforcement requests for their data, including emails, and other online activity. All these firms are in the process of updating their company policies to “expand routine notification of users about government data seizures,” despite the subpoenas explicitly stating the subject of the investigation should not be alerted.
In a statement, Facebook told DigitalTrends: "We are committed to transparency, and providing notice about government requests is an important part of being transparent. We are always working to improve our notification process as the law permits."
According to the Washington Post, the Justice Department isn’t pleased with the mini rebellion, and claims by notifying customers of such activity could not only put the subject in danger, but also risk ruining active criminal investigations. The companies say that people have a right to know when their data is being targeted, and this gives them the chance to take the battle to court, should they not want their privacy invaded.
While the majority of us will initially side with the tech firms on this issue, the Justice Department does make a compelling argument for everyone to keep their mouths shut. The report quotes an official statement, saying investigations could be threatened, and “potential crime victims” could be put in danger.
Department spokesperson Paul Carr lists the destruction of evidence, intimidation of witnesses, and fleeing suspects as some of the potential risks, adding these things are “unfortunately routine” in situations where people are suddenly made aware of surveillance. A former FBI agent agreed, damning the tech firm’s decisions as PR exercises at the "expense of public safety."
However, Google told Digital Trends: "We notify users about legal demands when appropriate, unless prohibited by law or court order." It was added that notifications would be sent to users except in specific situations, such as when there was a danger of death or serious physical injury to a person. 
Not every rule will be broken either, and officials from the aforementioned companies have said they won’t reveal everything. The exceptions that will remain secret include requests sent by the Foreign Intelligence Surveillance Court, and administrative subpoenas sent by the FBI, all of which are covered by law. Data requests with a court-approved gag order would also be kept under the firms’ collective hats, which confirms Google's statement on the matter.
It’s a tech-land lawyer at Perkins Coie, quoted in the Posts’s report, who highlights the reason why the policy changes should be considered a positive move. “It serves to chill the unbridled, cost-free collection of data,” he said.
Update: Added quotes from Facebook and Google.
[Image courtesy of Rena Schild/Shutterstock]

Read more