Skip to main content
  1. Home
  2. Mobile
  3. News

Obama restricts NSA telephone metadata collection program, calls for more changes

Andrew Couts
By
obama announces overhaul nsa telephone data collection screen shot 2014 01 17 at 11 15 39 am

Seven months after the first disclosures from Edward Snowden went public, President Barack Obama announced Friday a list of “concrete and substantial reforms” to the way the United States conducts its surveillance operations. Changes include extending privacy protections to citizens in foreign countries, limiting the secrecy surrounding the use of National Security Letters, and reigning in the National Security Agency’s controversial bulk collection of telephone metadata.

NSA telephone metadata collection

The most significant changes for Americans concern the NSA’s telephone metadata collection program. Obama reiterated that the program, carried out under Section 215 of the PATRIOT Act, “does not involve the NSA examining the phone records of ordinary Americans” and has repeatedly been renewed by Congress. But he admitted that “ it has never been subject to vigorous public debate.”

Recommended Videos

“I am therefore ordering a transition that will end the Section 215 bulk metadata program as it currently exists, and establish a mechanism that preserves the capabilities we need without the government holding this bulk metadata,” he said.

Related

Effective immediately, NSA analysts will no longer be permitted to “query” its database of telephone metatada without first receiving permission from the Foreign Intelligence Surveillance Court (FISC). And queries must now be limited to two steps from a “phone number associated with a terrorist organization,” Obama said. Earlier limits restricted queries to three steps from a targeted number, a jump many critics saw as overly broad.

Moreover, Obama said he may follow a presidential review board’s recommendation that the federal government turn over storage of telephone metadata to a third party, but admitted that doing so could cause new problems.

“The Review Group recommended that our current approach be replaced by one in which the providers or a third party retain the bulk records, with the government accessing information as needed. Both of these options pose difficult problems,” Obama said. “Relying solely on the records of multiple providers, for example, could require companies to alter their procedures in ways that raise new privacy concerns. On the other hand, any third party maintaining a single, consolidated database would be carrying out what is essentially a government function with more expense, more legal ambiguity, and a doubtful impact on public confidence that their privacy is being protected.”

Obama has asked intelligence officials and the attorney general to submit recommendations for how to properly transfer storage of the telephone metadata database to a third party before the collection program comes up for reauthorization on March 28.

National Security Letters

Obama also announced plans to limit the FBI’s use of National Security Letters, which often forbid recipients from disclosing that they had received the letter due to national security concerns.

“I have … directed the Attorney General to amend how we use National Security Letters so this secrecy will not be indefinite, and will terminate within a fixed time unless the government demonstrates a real need for further secrecy,” Obama said. He also plans to allow companies, like Google or Facebook, to “make public more information than ever before about the orders they have received to provide data to the government.”

Limit foreign intelligence gathering

Obama also issued a presidential directive that he says “will clearly prescribe what we do, and do not do, when it comes to our overseas surveillance.”

“In terms of our bulk collection of signals intelligence, US intelligence agencies will only use such data to meet specific security requirements: counter-intelligence; counter-terrorism; counter-proliferation; cyber-security; force protection for our troops and allies; and combating transnational crime, including sanctions evasion,” Obama said. “Moreover, I have directed that we take the unprecedented step of extending certain protections that we have for the American people to people overseas.”

Additional efforts will be taken to ensure that “unless there is a compelling national security purpose … we will not monitor the communications of heads of state and government of our close friends and allies.”

Review technological capabilities

Finally, Obama has ordered “a comprehensive review of big data and privacy” by a group that includes both government officials and “technologists and business leaders,” which will “look at how the challenges inherent in big data are being confronted by both the public and private sectors; whether we can forge international norms on how to manage this data; and how we can continue to promote the free flow of information in ways that are consistent with both privacy and security.”

More work to do

While Obama says that the reforms announced today “will point us in a new direction,” he admits that “more work will be needed in the future.”

“One thing I’m certain of: this debate will make us stronger. And I also know that in this time of change, the United States of America will have to lead,” Obama said. “It may seem sometimes that America is being held to a different standard, and the readiness of some to assume the worst motives by our government can be frustrating. No one expects China to have an open debate about their surveillance programs, or Russia to take the privacy concerns of citizens into account. But let us remember that we are held to a different standard precisely because we have been at the forefront in defending personal privacy and human dignity.”

What do you think of Obama’s efforts to curtail surveillance activities? Tell us down below.

Editors' Recommendations

Topics
Andrew Couts
Andrew Couts
Former Digital Trends Contributor
Features Editor for Digital Trends, Andrew Couts covers a wide swath of consumer technology topics, with particular focus on…
NSA review board proposes 46 changes to US surveillance practices
NSA headquarters

Times, they may be a’ changin’ – at least as far as the National Security Agency’s spying activities are concerned.
The White House today released the full report (pdf) from the presidential NSA review panel, which recommends dozens of changes to the way the spy agency conducts its operations. If adopted, the recommended polices would provide greater oversight and accountability, and aim to restore Americans’ trust in both the US intelligence community and US-based technology companies.
End NSA’s bulk phone metadata collection
For average Americans, the most significant recommendation may be changes in the bulk collection of telephone metadata, which was recently deemed likely unconstitutional by a federal judge who called it “almost-Orwellian.” Rather than allow the NSA to continue its practice of collecting and storing the metadata of virtually every phone call made in the United States, the review panel believes private companies should collect the data, and only make it accessible to the NSA under court order.
“In our view, the current storage by the government of bulk meta-data creates potential risks to public trust, personal privacy, and civil liberty,” wrote the panel. “We recognize that the government might need access to such meta-data, which should be held instead either by private providers or by a private third party. This approach would allow the government access to the relevant information when such access is justified, and thus protect national security without unnecessarily threatening privacy and liberty.”
Restrict US government’s demands for private data
Additionally, the review board proposes “important restrictions” on the ability of the Foreign Intelligence Surveillance Court (FISC), which oversees the NSA’s surveillance activities in secret, to “compel third parties (such as telephone service providers) to disclose private information to the government.” Further, it endorses tighter restrictions on the FBI’s used of National Security Letters (NSLs), which require individuals or organizations to hand over private information while often times forbidding the recipients from speaking about government’s demands. The panel says NSLs should only be used with “prior judicial review except in emergencies, where time is of the essence.”
Stop breaking encryption and installing backdoors
The panel also suggests that the NSA halt its efforts to weaken commercial encryption standards, like PGP and others, that are used to protect communications and data store in the cloud. It further advises the NSA to stop attempts to discover or create so-called backdoors into commercial technology products to allow for agency intrusion; and to end its attempts to discover vulnerabilities in commercial software for cyberattack purposes, as a way to help rebuild trust in US-made computer programs.
Increase transparency and public participation
In another move that poised to appease US technology companies, many of which have demanded reforms of the NSA, the panel calls for greater transparency concerning government requests for personal data. The changes include allowing companies like Google or Facebook to divulge more information about the number and types of requests they receive from the government, and the number of people affected by these requests. The federal government is also asked to issue regular reports about its requests for citizens’ data.
Importantly, the panel further endorses transparency though a "public interest advocate to represent the interests of privacy and civil liberties" in arguments before the FISC. While these cases would be classified – meaning the public would not likely have access to the court records – it would be a marked increase in the ability of the public to have a say in the ways our rights are affected by the NSA's activities.
Moving forward
The 300-page report includes a total of 46 recommendations that would, if enacted, affect the privacy of both US citizens and non-US persons, whom the panel believes should receive the same protections provided by the Privacy Act of 1974 as Americans.
It is not clear which, if any, of these proposed policy changes the government will enact. As The New York Times reports, some of the changes can be instituted by President Obama alone, while others require congressional action. One of the most fundamental proposed changes in the NSA’s structure – to split command of the NSA between different agencies – has already been rejected by Obama.
Even if all of the changes are adopted, the NSA's surveillance activities will continue.
Read the panel’s full report below:
NSA Review Board Final Report: LIBERTY AND SECURITY IN A CHANGING WORLD

Read more
5 things you need to know about today’s historic ruling against NSA surveillance
National Security Agency Director General Keith Alexander

A federal judge ruled Monday that the National Security Agency’s bulk collection of phone call data in the US is likely unconstitutional – and that’s a big deal. The decision is being heralded by privacy advocates as a "historic" first step in a long, long staircase that could potentially end with a Supreme Court ruling against the NSA’s dragnet collection of all Americans’ phone call records. But, of course, it’s a lot more complicated than that. Here are the key facts every American needs to know.
1. The NSA’s phone data collection won’t stop – at least not yet
Judge Richard Leon of the US district court for the District of Columbia found (pdf) that the plaintiffs in the lawsuit “demonstrated a substantial likelihood of success on the merits of their Fourth Amendment claim.” This basically means that Judge Leon believes there’s a good chance this case against the NSA’s practice of collecting so-called metadata – numbers called, call duration, call times, and more – violates the Constitution’s protection against unreasonable searches and seizures. Judge Leon also found that the plaintiffs “will suffer irreparable harm” if the NSA’s bulk phone data collection doesn’t stop. This estimation of harm ostensibly applies to every American – not just the plaintiffs in the case – since anyone who makes or receives a phone call in the US is said to have had his or her phone metadata sucked up by the NSA.
“I cannot imagine a more ‘indiscriminate’ and ‘arbitrary’ invasion than this systematic and high-tech collection and retention of personal data on virtually every single citizen for purposes of querying and analyzing it without prior judicial approval,” Judge Leon wrote. “Surely, such a program infringes on ‘that degree of privacy’ that the founders enshrined in the Fourth Amendment.”
Despite his strong language, Judge Leon also decided that, due to the “significant national security interests at stake” and the unique nature of the case, he has put his order to stop the NSA surveillance program on hold until the government can file an appeal.
2. The NSA cannot prove that mass phone data collection has stopped any terrorist attacks
In addition to finding that the NSA’s phone metadata collection is harmful to Americans and likely unconstitutional, Judge Leon also found zero evidence that it has proven crucial in stopping even a single terrorist attack.
“Given the limited record before me at this point in the litigation – most notably, the utter lack of evidence that a terrorist attack has ever been prevented because searching the NSA database was faster than other investigative tactics – I have serious doubts about the efficacy of the metadata collection program as a means of conducting time-sensitive investigations in cases involving imminent threats of terrorism,” he wrote.
Under this reasoning, Americas’ constitutional right to privacy outweighs the government’s interest “in collecting and analyzing bulk telephony metadata” – so it should stop.
3. Technology may render the US government’s legal justification for NSA phone data collection irrelevant
The Obama administration maintains that the secret Foreign Intelligence Surveillance Court (FISC) has found that dragnet phone metadata collection is constitutional. This status is heavily based on a 1979 Supreme Court decision, Smith v. Maryland, which found that police could collect a few days’ worth of suspects’ phone metadata without a warrant because the telephone company – a third party – also had access to that data, meaning there was no expectation of privacy on the suspects’ part.
Judge Leon blasted this justification, writing that the “almost-Orwellian technology that enables the government to store and analyze the phone metadata of every telephone user in the United States is unlike anything that could have been conceived in 1979.” And, because the Smith ruling involved “highly-limited data collection” whereas the NSA’s surveillance program involves at least five years’ worth of data from many more phones, the ruling cannot be used to justify the latter practice.
“Admittedly, what metadata is has not changed over time. As in Smith, the types of information at issue in this case are relatively limited: phone numbers dialed, date, time, and the like,” wrote Judge Leon. “But the ubiquity of phones has dramatically altered the quantity of the information that is now available, and more importantly, what that information can tell the Government about people’s lives.”
Read more about the NSA's Big Data technologies here.
4. The plaintiffs in the case aren’t your average citizens
The case at hand was brought forth by two Verzion customers: Larry Klayman and Charles Strange. But unlike many of us, this pair may be uniquely suited – or, perhaps, ill-suited – to challenge the constitutionality of the NSA’s phone metadata collection.
Klayman is a conservative activist lawyer with a long history of challenging the federal government. In the 1990s, he filed multiple lawsuits agains the Clinton administration over various allegations of professional and personal “transgressions,” reports The New York Times. He also took former Vice President Dick Cheney to task over secretive energy meetings. And he unsuccessfully challenged President Barack Obama’s placement on Florida’s primary ballot due to allegations that the Commander in Chief is not a natural-born US citizen.
Charles Strange, meanwhile, is the father of the late Michael Strange, a cryptologist who worked for the NSA and provided support to the Navy SEAL Team 6 that killed Osama bin Laden. Michael was killed in August 2011 after a helicopter he was riding in was shot down by Taliban forces.
Klayman also represents the Strange family and other families of US military personnel who died in the crash in a case against Vice President Biden and Secretary of Defense Leon Panetta, who the plaintiffs claim are responsible for their sons’ deaths for disclosing the soldiers’ identities.
5. Snowden sees Judge Leon’s decision as victory
Edward Snowden, the former NSA analyst whose leak of classified agency documents made public the agency’s mass phone metadata collection, made a rare public statement following Judge Leon’s decision, and heralded it as justification for his actions.
“I acted on my belief that the NSA’s mass surveillance programs would not withstand a constitutional challenge, and that the American public deserved a chance to see these issues determined by open courts,” Snowden said in a statement distributed by journalist Glenn Greenwald, to whom Snowden first provided the NSA documents. “Today, a secret program authorized by a secret court was, when exposed to the light of day, found to violate Americans’ rights. It is the first of many.”
Read Judge Leon's full 68-page ruling below:
Klayman v. Obama: NSA mass phone metadata collection ruled likely unconstitutional

Read more
What’s the NSA picking out of your phone calls? Just ‘unvolunteered truths’
National Security Agency Director General Keith Alexander

“The first program executed under Section 215 of the Patriot Act authorizes the collection of telephone metadata only,” said National Security Agency Director General Keith Alexander before the House Intelligence Committed on June 18. Called for a rare public hearing on the NSA’s activities, Alexander uttered this statement just a week after the first of an ongoing set of top-secret leaks from former contractor Edward J. Snowden.
“As you've heard before, the metadata is only the telephone numbers, and contact, the time and date of the call, and the duration of that call,” Alexander continued. “This authority does not, therefore, allow the government to listen in on anyone's telephone calls, even that of a terrorist.”
Predictive analytics is about revealing “unvolunteered truths” about people.
This is one of the prevailing storylines of the NSA and its defenders: The phone data collected, via Verizon, on virtually every man, woman, and phone-using child in the United States is nothing to fret about. As James Cole, the U.S. Deputy Attorney General put it during the same hearing, this metadata is exactly what “you show to many, many people within the phone company on a regular basis.” And yet, it is this same phone data that the NSA describes as crucial to its counterterrorism efforts.
Let’s put aside for a moment the question of whether this dragnet data collection should happen, and ask a crucial question now that we know that it is: How is it possible that our phone metadata can be both innocuous and key to preventing, say, a group or individual from releasing poisonous gas in the New York City subway system?
A detailed answer to that question is classified. But based on what we know about the techniques available, an educated look at the possibilities points us in one direction: Predictive analytics.
Predictive analytics 101
Put entirely too simply, predictive analytics is a way of using “Big Data” techniques to predict possible outcomes, from results of presidential elections to the behavior of individual people. Depending on the application, predictive analytic data often includes demographic information, family and marital status, purchasing history, past weather patterns, business transaction histories, social media activity, website clicks, and, of course, telephone metadata, all of which help shape the map of an possible underlying reality.
“It’s about creating the ability to not only predict the future, but to influence it,” says Dr. Eric Siegel, author and founder of the Predictive Analytics World conferences. Further, he explains, it’s about revealing “unvolunteered truths” about people.
We know the NSA uses a variety of Big Data techniques and technologies, some of which the agency itself developed. The question then is: How are they using computer-assisted prediction?
How we use predictive analytics
The public may have first become aware of predictive analytics in 2012, thanks to an article in The New York Times Magazine by Charles Duhigg. You may remember its bombshell anecdote: A father, having discovered coupons from Target offering discounts on baby gear sent to his young daughter, became outraged at the big box retailer for apparently trying to coerce the teenager into having sex. In fact, Target’s in-house data analysts had devised an algorithm that deduced whether particular customers were pregnant based on seemingly random changes in their purchasing habits.
The man later apologized to a Target manager; his daughter, he had since discovered, was due a few months later.
Dr. Eric Siegel, founder of Predictive Analytics World conferences
This is, to date, the most visceral story we have about the powers of predictive analytics to derive facts about the world through data points and algorithms. But the fruits of predictive analytics are all around us, in less creepy forms: Netflix’s video recommendations, email spam filters, and Google Now, the technology giant’s virtual “personal assistant” for mobile devices, are all prime examples of predictive analytics at work. Rather than accurately predicting your due date, Google Now simply suggests a less congested route to your office.
In his aptly titled new book, Predictive Analytics: The Power to Predict Who Will Click, Buy, Lie or Die, Siegel lays out multiple poignant examples of how predictive analytics is predicting more than just which customers to send ads.
Hewlett-Packard, for instance, has developed a “Flight Risk” score for its employees that “foretells whether you’re likely to leave your job,” writes Siegel. Credit card companies accurately decipher which of us will make late payments. Parole boards figure out which inmates are most likely to commit another crime upon release. Cell phone companies figure out who and when to serve offers for discounted phones. The U.S. military uncovers which soldiers can handle the highly demanding life in the Special Forces. Life insurance providers predict when people will die. The list goes on – and is getting bigger by the day.
Future crime
In recent years, predictive analytics has entered a new realm: Policing. The highest-profile example of so-called “predictive policing” began in Santa Cruz, California, in 2011. Santa Clara University professor George Mohler generated predictions of crime “hot spots” based on historical crime data dating back to 2006, as well as new data added to the system every day. The result of that experiment is a propriety software called PredPol, created in 2012 by Mohler and P. Jeffrey Brantingham of UCLA, which has since been adopted by an increasing number of police departments around the country, including the Los Angeles Police Department.
While the term “predictive policing” inevitably conjures scenes from Minority Report, the reality has far more in common with the Waze traffic app than a dystopian future envisioned by three sun-deprived creatures in a vat of KY Jelly. In other words, PredPol does not let officers predict who will commit a crime, but where a crime will likely happen, regardless of who commits it.
PredPol is designed to be “completely blind to people,” says Zach Friend, a Santa Cruz district supervisor and police consultant, who spearheaded the development of PredPol. “The only four inputs into the system are date, time, crime type, and location.”
An example of the PredPol user interface.
Police have long used “hot spot” mapping tools, which rely upon historical crime data to help departments pinpoint problem areas. PredPol is the next evolution in hot spot policing. Think of it as a traditional hot spot map that’s updating in real time and deploying officers to specific locations at specific times where and when a specific crime is deemed likely.
Crime data can often be sent to patrol cars as it comes in from victims, giving officers a birds eye view of the surrounding criminal landscape, as it evolves minute by minute.
Powerful though predictive policing may be, the limitation imposed on the types of data used in crime prediction is a key component to preventing PredPol from edging into dangerous territory, says Friend.
“I wouldn’t want to say that ‘Andrew’ is going to be [committing a specific crime],” says Friend. “That’s the Minority Report stuff. What we wanted instead is, ‘This location, this [latitude and longitude], 500-foot by 500-foot, has the highest probability of a burglary occurring during this shift. That’s all we wanted to do.”

The result of this tactic is a clear and continuous drop in crime. Santa Cruz saw burglaries drop by 19 percent in 2012, compared to the same period in 2011. And Los Angeles saw a 12 percent crime reduction soon after adopting PredPol. For all of 2012, overall crime in L.A. dropped by 1.4 percent, and violent crime dropped 8.2 percent.
While PredPol may be committed to limiting its system to time and location predictions, other data scientists have begun to take the technology a step further. According to Bloomberg News, Jim Adler, a former chief privacy officer for data broker Intelius, has created a computer model that can “accurately” predict whether an individual will commit a felony. Adler’s software initially made such predictions using a small number of telling data points, including gender, eye and skin color, traffic citations, criminal history, and “whether the individual has tattoos,” according to Bloomberg.
Predictions and the NSA
The rise of PredPol and other predictive policing efforts opens a window into what kinds of predictive tools the NSA may have at its disposal, and possibly how the spy agency uses those tools. For example, during the wars in Iraq and Afghanistan, the NSA utilized predictive models and other Big Data tools created by Silicon Valley company Palantir, to help connect the dots between known terrorists. According to government officials, Palantir’s technology even helped the military foil suicide bomber and roadside bomber attacks. Similar tools from Palantir are now used by the NSA, FBI, and CIA, which provided monetary backing to the company.
Some of the most powerful Big Data weapons were developed by the NSA itself. Chief among these is a system called Accumulo (pdf), which enables the NSA to map trillions of “nodes” (data points) and “edges” (connections between two or more data points), to create a discernable picture of the world’s communications using telephone metadata and internet traffic data collected under its now-controversial PRISM program.
According to a former international law enforcement agent, who requested anonymity due to ongoing professional obligations, the NSA can likely pinpoint suspicious communications activity using a predictive model, based on call records and Internet activity. The predictions may include deriving name, age, and gender data, or plotting likely future locations of individual people.
NSA headquarters at Ft. Meade, Maryland
“Predictive analysis can often provide an idea of a subject's age and gender based solely on call frequency and duration,” says the former agent. “Call duration and call frequency can be indicators of the significance of the callers to each other.”
In the same vein, geographic location data can be used “to make inferences about how criminal organizations operate,” says the agent. The same goes for terrorist groups, and the information can be used to deduce who is, and is not, working together on a potential plot, and where these actors might meet.
“Predictive analysis might indicate when it's likely someone will show up at a particular locations,” says the agent. “Similarly, if a model gives a high probability that two individuals will meet in a time window and that window elapses, it can be an indication something has changed in their relationship. That might just be a flat tire, or it might be a falling-out.”
This information may be more valuable than direct surveillance – it’s spying 2.0
The inferences that can be obtained from predictive analytics don’t stop at who knows whom, however. According to Siegel, metadata, when fed into a well-tuned predictive model, can reveal more about a person and their intentions than a particular email or telephone conversation can. That is to say, predictive analytics allows the NSA or law enforcement agencies to derive actionable intelligence about targets that those targets have not disclosed, as well as information that may be more valuable than direct surveillance – it’s spying 2.0.
“All these metadata things are the nuts and bolts – the sort of syntactic things known [about a person] that are often extremely revealing, not to mention a lot easier to deal with than the content, such as the typed message of an email or spoken words during a phone call,” says Siegel.
Siegel points out that, in any particular case, what a target says online or over the phone may prove to be “the most important of all [for an investigation], but are also not necessarily going to trump the metadata. And the metadata itself is going to make a huge difference.”
Metadata is so revealing because it is “easy to aggregate and analyze” in its raw form, according to Princeton Computer Science Prof. Edward Felten, in a recent court filing (pdf) written on behalf of the American Civil Liberties Union, which is suing (pdf) top Obama administration officials over the NSA’s collection of telephone metadata. Speech and the meaning of emails are, by contrast, much more difficult for computers to decipher, explains Felter. Add to that the significant increase in computing power and the plummeting cost of storage, and it’s possible for the NSA to easily garner “sensitive details about our everyday lives.”
The NSA's call metadata collection includes information on millions of Americans bikeriderlondon/Shutterstock
“Sophisticated computing tools permit the analysis of large datasets to identify embedded patterns and relationships, including personal details, habits, and behaviors,” writes Felten. “As a result, individual pieces of data that previously carried less potential to expose private information may now, in the aggregate, reveal sensitive details about our everyday lives – details that we had no intent or expectation of sharing.”
Compare this to the contents of a telephone call, which requires far more legwork to shape into a usable form. To analyze phone calls, “the government would first have to transcribe the calls and then determine which parts of the conversation are interesting and relevant,” explains Felten. “Assuming that a call is transcribed correctly, the government must still try to determine the meaning of the conversation: When a surveillance target is recorded saying ‘the package will be delivered next week,’ are they talking about an order they placed from an online retailer, a shipment of drugs being sent through the mail, or a terrorist attack? Parsing and interpreting such information, even when performed manually, is exceptionally difficult.”
To illustrate the powers of predictive analytics further, Felten lays out the following hypothetical:
“A young woman calls her gynecologist; then immediately calls her mother; then a man who, during the past few months, she had repeatedly spoken to on the telephone after 11pm; followed by a call to a family planning center that also offers abortions. A likely storyline emerges that would not be as evident by examining the record of a single telephone call.”
NSA's National Security Operations Center (NSOC), considered the center of the agency's surveillance efforts Wikipedia
Expand the telephone metadata collect over a period of months or years, and “many of these kinds of patterns will emerge once the collected phone records are subjected to even the most basic analytic techniques,” writes Felten. One can easily imagine how these techniques could be used to root out terrorists – or abused to spy on innocent people.
While we do not know the extent to which the NSA employs predictive modeling, officials have confirmed that it uses analytics for purposes similar to those outlined by Felter. As David Hurry, chief researcher for the NSA’s computer scientist division, told InformationWeek in 2012, “By bringing data sets together, it’s allowed us to see things in the data that we didn't necessarily see from looking at the data from one point or another.”
Limitations of prediction
As powerful a tool as predictive analytics may be, its proprietors are quick to point out that it is not a silver bullet or a magic crystal ball.
“Generally, it’s not about accurate predictions,” says Siegel. “It’s about predicting significantly better than guessing. And that’s true for most applications, including law enforcement applications.”
Due to the limitations of predictive analytics, experts believe the NSA likely uses this Big Data technique as a jumping off point, or to enhance other parts of ongoing investigations.
“In my experience predictive modeling alone is not used to try to identify suspects or persons of interest,” says our unnamed source. “I can't say that it has never been used that way. I have, however, used predictive modeling techniques in the course of medium- and long-term investigations opened for other reasons."
Predictive analytics lies at the heart of the NSA's surveillance efforts
Dean Abbott, founder of Abbott Analytics, says that the telephone metadata collected by the NSA may be “particularly useful in connecting high-risk individuals with others whom the NSA may not know beforehand are connected to the person of interest.” He adds, however, that predictive tools are likely only helpful when combined with a variety of other intelligence.
“These networks are usually best used as lead-generation engines,” writes Abbott on the Predictive Analytics World blog. Used this way, Abbott explains, predictive analytics and link analysis, such as the connections revealed through Accumulo, are “tremendously powerful” tools.
Drawing the line
Limited though predictive analytics may be, experts believe the potential for the technique to transform the nature of privacy and civil liberties is potent.
It is “fairly easy to predict” whether a criminal will repeat his pattern of crime, says John Elder, CEO of Elder Research Inc., the largest dedicated data analytics company in the U.S.. “Now, do you do like that Tom Cruise movie where you intercept somebody before they do the crime? That’s probably going too far in terms of allowing humans to take responsibility for their own actions.”
Big Data systems can also be tailored with privacy concerns built in. Accumulo, for example, has complex tagging system, known as Column Visibility, which the NSA built in to limit, at every level, which analysts can see what types of data through the use of security labels. It is likely that this aspect of Accumulo is what the NSA means when it refers to built-in civil liberty and privacy protections.
“Individual pieces of data … may now, in the aggregate, reveal sensitive details about our everyday lives.”
Some legal experts believe predictive modeling will force us to overhaul our privacy laws. In an expansive analysis of the legal implications of predictive policing published in 2004, University of the District of Columbia Assistant Professor of Law Andrew G. Ferguson, concludes that predictive analytics will change the ways in which police, such as the FBI or LAPD, apply the Fourth Amendment’s guarantee that law enforcement authorities must have “probable cause” before engaging in a search and/or seizure.
“… Predictive policing forecasts will end up being seen as a ‘plus factor’ to find reasonable suspicion,” writes Ferguson. “However, the use of predictive policing forecasts alone will not constitute sufficient information to justify reasonable suspicion or probable cause for a Fourth Amendment seizure.”
In other words, by Ferguson’s assessment, predictive modeling may be used to unearth our innermost secrets – but the courts likely won’t let police search us on that intelligence alone.
For now, the debate over the NSA’s surveillance remains firmly focused on the collection of the data, not the ways in which it is likely used to derive more intelligence. In July, the U.S. House of Representatives narrowly defeated a bill that would have largely cut off the NSA’s ability to collect telephone metadata not directly connected to an ongoing investigation. And now, following recent revelations that the NSA violated privacy protection rules thousands of times in 2012, legislators are ramping up efforts to push back against the NSA with at least three new bills.
While we may see limitations on data collection by the NSA, the use of predictive analytics to help weed out criminals and terrorists – or even just to sell us diapers and cell phones – isn’t going anywhere, says Siegel.
“The power of predictive analytics should not be underestimated in the same way that the power of a knife should not be,” he says. “A knife could be used for good or for evil. I think outlawing knives entirely is not on the table.”

Read more