Skip to main content

The birth of big data: How Simulmatics predicted the future 60 years ago

Technicians evaluate the UNIVAC computer system in 1951, which took up 352 square feet of floor space and ran at a then-astronomical rate of 2.25 megahertz. Getty

In the 1957 film Desk Set, Katherine Hepburn plays a reference librarian whose job is seemingly threatened by a “mechanical brain,” a room-sized computer named EMERAC. Though she assures her fellow researchers that no machine can do their jobs, her co-worker replies, “that’s what they said in payroll,” before half the workers were fired.

Two years after that movie premiered, a new company was founded, amid the uneasiness around automation. The Simulmatics Corporation, as it was called, was a consulting firm that promised with enough data, it could sway voters’ opinions and predict the outcome of elections. It also claimed it could apply the same methods to advertising, getting more people to buy a certain brand of soap or breakfast cereal.

If you’ve never heard of Simulmatics, it’s because it didn’t really succeed in its grand plans. But according to historian Jill Lepore, author of If Then: How the Simulmatics Corporation Invented the Future, its legacy lives on: “By the early twenty-first century, the mission of Simulmatics had become the mission of many corporations, from manufacturers to banks to predictive policing consultants. Collect data. Write code. Detect patterns. Target ads.”

A crack team

Simulmatics was up and running in early 1959 and bankrupt by August 1970. In the interim, it supplied reports to John F. Kennedy’s election campaign, tried to predict an election for The New York Times, attempted to stop communism in Vietnam, and claimed to be able to predict when race riots would occur.

Lepore compares hiring for the company to assembling a team for a bank heist. There was social scientist Bill McPhee, computer scientist Alex Bernstein, and Ithiel de Sola Pool, a political scientist. Bringing them all together was Ed Greenfield, “the huckster,” according to Lepore. He’d become enthralled with the idea of marrying computers and politics ever since 1952, when CBS employed a UNIVAC (Universal Automatic Computer) — and a prop version for the TV studio — to predict the election’s winner.

Two men work at different components of a large Univac computer in an office. Photo by Hulton Archive / Getty Images

Using a variety of poll data — of mixed quality, according to one skeptic — Simulmatics created simulations of voting behavior to test out different campaign strategies. McPhee, Pool, and the rest of the team took data from thousands of Gallup and Roper surveys to create 480 voter types, like “Midwestern, rural, Protestant, lower income, female.” They also cataloged the groups’ views on issues like McCarthyism and the hydrogen bomb. It took ages to create all the punch cards with the information from the polls and surveys, but afterward, the Simulmatics group could mine the data for answers to any of their queries and receive the results in about 40 minutes, simulating how these types would respond to the candidate taking different stances.

If Kennedy wanted to win, he needed to push a strong civil rights platform and actively bring up his Catholicism. Any anti-Catholic backlash would sway those concerned about prejudice — like Jewish voters — toward Kennedy, according to Simulmatics. To explain what they were doing to potential clients, employees would compare it to weather forecasting: “One can predict tomorrow’s weather best if one has not only current information but also historical information about patterns into which current reports can be fitted.”

The pitch worked. The New York Times briefly contracted with Simulmatics to interpret the 1964 election results in “real time,” a fairly new concept that meant there wasn’t a huge lag between results coming in and the computer’s analysis.

Greenfield figured the secret sauce didn’t need to just apply to elections. He also courted brands, saying Simulmatics could make consumer profiles similar to voter types. The U.S. Department of Defense’s Advanced Research Projects Agency requested a proposal on Simulmatics’ methods for “dealing with the problems of counter-insurgency.”

The flimflam machine

Despite Greenfield’s expertise in selling Simulmatics, the results didn’t always deliver. How or if anyone used its reports is murky. It’s a bit like how most people probably viewed UNIVAC at the time: Feed in the punch cards, get a result. What happens in between is alchemy. The New York Times canceled its contract with the company after some buggy trials leading up to the election. Instead of partnering with the corporation and giving up their precious data — which Simulmatics needed to make its models work — advertising agencies like BBDO decided to build their own.

In the mid-1960s, Simulmatics went to Vietnam to, in Pool’s view, predict and stop insurgency before it happened. To accomplish this, the company bought over an American team and hired Vietnamese interpreters to administer lengthy questionnaires, written in English, to villagers. The Vietnamese workers found that the questionnaire didn’t make much sense when translated, and Simulmatics didn’t explain the main goal of the research, which was to “assess what changes in information, attitudes, and behavior result from placing TV sets in Vietnamese villages.” An assessment of the research pointed out that the question was possible to ascertain unless the company researched every facet of Vietnamese life. “So far from being a way to study all things about all Vietnamese, Simulmatics’ research appeared to some people at ARPA as a way to study nothing at all,” writes Lepore.

“The fantasy of computer-aided riot prediction endured; that ongoing civil unrest and racial inequality and police brutality can be addressed by more cameras, more data, and more computers, and, above all, by predictive, what-if algorithms.”

Even Simulmatics’ seeming successes were a bit dubious. “It is not known what, if any, influence the Simulmatics simulation had in the development of Kennedy’s approach to the religious issue after August 25,” Thomas Morgan wrote in a 1961 Harper’s Magazine article about the company, adding, “It seems that, at most, the simulation may have lent some psychological support to those Kennedy strategists who favored its conclusions anyway.”

“It is true that Simulmatics did contribute some reports to the campaign, but it’s worth emphasizing that, even without the company’s advice, Kennedy’s team was savvy enough to adopt tactics that increased his appeal to Black voters and openly confronted the issue of his Catholicism,” Stephen Schlesinger wrote in a letter to The New Yorker after an expert from If Then was published. He’s the son of the son of historian Arthur Schlesinger, Jr., who wrote A Thousand Days: John F. Kennedy in the White House.

The future is then?

There’s a scene in Desk Set where Katherine Hepburn’s character describes seeing a demonstration of IBM’s computer, with its ability to translate Russian into Chinese. “Gave me the feeling that maybe, just maybe, people were a little bit outmoded,” she says. “Wouldn’t surprise me a bit if they stopped making them,” Spencer Tracey responds.

“When a machine takes the job of 10 men, where do those 10 men go?” Kennedy asked in 1960. The fears of automation were already present. As Lepore points out, many of the early jobs that computers replaced belonged to women — secretarial tasks like typing and filing.

Morgan’s story in Harper’s Magazine caused a bit of a stir, as it was meant to do. He was actually Simulmatics’ PR person, in addition to being a freelance writer. But in the piece he raised all kinds of questions that people are still asking today: “As we seek more and more data for the machines, can we maintain our traditions of privacy?” he wrote.

Lepore describes the men of Simulmatics as “midcentury white liberals” who raised money for causes like civil rights. They started out well-intentioned, but those intentions looked very different in the villages of Vietnam and the streets of Rochester, New York, where they wanted to quell riots.

In a letter to his son, Pool wrote, “I’m privy to all kinds of information that you can’t possibly know. So there’s no point in arguing because your opinion is uninformed; we’re not coming from positions of equal knowledge.” Pool would go on to predict social networks, information bubbles, and the transition to storing everything from tax returns to school transcripts on computers. “Pool predicted so accurately because he knew so much,” writes Lepore.

But Pool’s attitude toward his son — that because he had a different experience, his point of view didn’t matter — endured, too. The problem is that Simulmatics, in trying to smother revolutions and riots, wanted to halt a chemical reaction instead of looking at the elements that caused it.

After a series of riots in the mid-1960s, President Lyndon B. Johnson appointed the Kerner Commission to investigate their causes. Simulmatics provided information on the media’s response to these riots. But the panel’s main finding was that housing discrimination, voter suppression, and education and employment inequalities were creating “two societies, one black, one white — separate and unequal.” The report establishing programs that would require  “unprecedented levels of funding and performance, but they neither probe deeper nor demand more than the problems which called them forth. There can be no higher priority for national action and no higher claim on the nation’s conscience.”

But Johnson didn’t provide that funding, and, as Lepore writes, “the fantasy of computer-aided riot prediction endured” in the belief “that ongoing civil unrest and racial inequality and police brutality can be addressed by more cameras, more data, and more computers, and, above all, by predictive, what-if algorithms.”

Editors' Recommendations

Jenny McGrath
Former Digital Trends Contributor
Jenny McGrath is a senior writer at Digital Trends covering the intersection of tech and the arts and the environment. Before…
British watchdog group orders Cambridge Analytica to hand over data
Crisis Response Hub

As the fallout of data firm Cambridge Analytica continues, a U.K. data privacy watchdog group has ordered that the company cede all of the personal information it collected from an American professor back to him, noting that even non-British citizens have the right to seek and obtain data held by a British firm. In doing so, the Information Commissioner's Office (ICO) has set a precedent for how Cambridge Analytica and other firms would have to deal with illegally collected information, potentially allowing millions of other U.S. Facebook users to demand information on exactly what data companies have on them.

In a notice posted on Saturday, the ICO said that it had "served a legal notice on SCL Elections Ltd," the parent company of Cambridge Analytica, "ordering it give an academic all the personal information the company holds about him." In total, the company will have 30 days to comply with the subject access request, as submitted by Professor David Carroll. If Cambridge Analytica does not do so, it will be violating the Data Protection Act of 1998, and will be committing a criminal offense "punishable in the courts by an unlimited fine."

Read more
Cambridge Analytica’s ex-director wants to fix data privacy. Can we trust her?
brittany kaiser cambridge analytica iovo feature stolen data


Brittany Kaiser, Executive Adviser to IOVO and former Director of Program Development at Cambridge Analytica. Leon Neal/AFP/Getty Images
(in)Secure is a weekly column that dives into the rapidly escalating topic of cybersecurity.
Data is valuable. In fact, some economists say data is the most valuable commodity in the world, even surpassing oil. That fact was quoted by ex-director at Cambridge Analytica, Brittany Kaiser, who spoke on April 18, 2018, at a press conference in New York for a new initiative on data privacy. The organization she’s advising is called IOVO, the Internet of Value Omniledger, and it goes against nearly everything she worked for at Cambridge Analytica.

Read more
Localblox data breach is the latest nightmare for Facebook, LinkedIn
what facebook users should know about cambridge analytica and privacy mobile v1

After Facebook found itself embroiled in the Cambridge Analytica data scandal that affected the personal information of 87 million of its users, the company is once again tied to another data breach. This time, Localblox is the culprit.

Like Cambridge Analytica, Localblox creates profiles of individuals using information scraped from publicly accessible sources, like social network profiles on LinkedIn, Facebook, Twitter, and Zillow. Localblox chief technology officer Ashfaq Rahman describes the process to ZDNet as creating transformative intelligence by joining bits and pieces together. A listing on Crunchbase describes Localblox as "a location-based social network that builds scalable neighborhood platforms, aggregating business profiles with metadata."

Read more