Big Data Wins the Biggest Prize

When Rush Limbaugh made fun of the Obama campaign dinner at Sarah Jessica Parker's townhouse, I laughed right along. It was the apotheosis of the Hollywood love-in for President Obama, ripe for mockery. It turns out that the Democrats were playing chess while the Republicans thought the game was checkers (the way things turned out a cute dog might have helped). Key to the President's re-election was

a massive data effort that helped Obama raise $1 billion, remade the process of targeting TV ads and created detailed models of swing-state voters that could be used to increase the effectiveness of everything from phone calls and door knocks to direct mailings and social media.

The dinner with Sarah, and its West Coast counterpart dinner with George Clooney, were not affectations but part of a well-thought-out strategy [bold added]:

For the general public, there was no way to know that the idea for the Parker contest had come from a data-mining discovery about some supporters: affection for contests, small dinners and celebrity. But from the beginning, campaign manager Jim Messina had promised a totally different, metric-driven kind of campaign in which politics was the goal but political instincts might not be the means. “We are going to measure every single thing in this campaign,” he said after taking the job. He hired an analytics department five times as large as that of the 2008 operation, with an official “chief scientist” for the Chicago headquarters named Rayid Ghani, who in a previous life crunched huge data sets to, among other things, maximize the efficiency of supermarket sales promotions.

Michael Scherer's article in Time is a fascinating read. Comments:

1) The Republicans undoubtedly will create, if they haven't done so already, a similar big-data operation.

2) The first important step, which began shortly after 2008, was the 18-month-long project to create a "mega" file that unified the many disparate Democratic databases. This means that work on 2016 has already begun (sigh).

3) Holding the Presidency confers an enormous advantage. The incumbent party--whether Democrat or Republican--will have access to oceans of data not necessarily available to its opponent. It is nigh impossible to enforce privacy rules, especially since enforcement is done largely through self-policing. [Flashback: Big Data is the new Plastics.]