It is a fascinating study in group psychology – how quickly the perception of 538 Nate Silver and his fellow psephologists changed from November 6 to November 7. In late October, the evisceration of Silver was in full swing. Mere weeks later, the upper hand had swung 180 degrees to where Silver and Sam Wang were heralded as heroes beckoning a new era of data driven prediction. As the current story line goes, the wonks’ data driven predictions will replace the anachronistic anecdotally driven predictions of the political punditocracy who have presence in the media more through their network contacts and past prestige than the efficacy of their insights.
When the evisceration was in full swing in October, the timing was quite odd. It seemed very premature – especially with only two weeks left in the campaign. If the attack had happened in June or July, it would have made more strategic sense. As it was, the stink of desperation hung in the air. With two weeks to go in the contest – the common and sensible response was “why not let it ride and see what happens?” Of course, the Republicans feared Silver’s prediction was becoming a self-fulfilling prophesy. The problem was that they waited too long before trying to ‘take out’ the prophecy. It would have been interesting if the Republicans had tried to take on Silver and the wonks earlier in the campaign cycle, i.e. back in June or early July. Silver’s numbers did not change dramatically between June and October – it was only the Republican’s perception of the importance of Silver’s prediction that changed. In fact, it was the state polling numbers that stirred up the Republicans ire in the first place, and Silver (with state polling being an important piece of his psephology methodology) was the next logical target once they started worrying about these poll numbers.
In general, the data back story of Election 2012, from the pre first debate polling conspiracy (pushed by Limbaugh and Hannity among others) to Karl Rove’s stubbornness about the Ohio call in Election night, was a totally wild ride. What has become clear in the process is that data is now a critical component of journalism, especially as it relates to politics and elections. To the wonk class, it has been clear for a long time that data helps us all to be better informed citizens. Data points need to be an integral part of the stories we are ingesting as citizens. Wonks around the world will be toasting Nate Silver and Sam Wang for awhile yet, and are probably pleased that such wonkery has now been introduced to the general population – and secretly pleased that such public acceptance rises to the level of making movies about Silver.
That said, data has almost become a useless term in its own right. We can’t be worried about data for data’s sake. We must refer to “highly nuanced data” within a clarified context. It is this data that, when captured and processed in real time, is creating the quantum leap forward in understanding and the predictive sciences. It is clear, when one listens to either Silver when he talks about he predicted the election, or Sam Wang when he talks about the future of election polling, that the age of the uninformed-by-data pundit has come and gone. The future belongs to those who generate insights sitting on top of processed data.
Meanwhile, the Republican’s awareness of Nate Silver’s five three eight blog as being a potentially self-fulfilling prophecy of Election 2012 raises a very interesting notion about the future of psephology. Silver’s prediction was based on accurately processing data (polling data, economic data, etc.). In a very simplistic sense, Silver is measuring the sentiment of a certain population (polls) under certain conditions (economic data). As Silver writes in his book, “The Signal and the Noise”, his methodology abides by probability, consensus, and a recognition that polls are constantly changing. What Silver’s method does not do is predict how these polls will shift over time or what drives these shifts in polls. Silver writes how the news coverage that truly affect a campaign proceeds at an irregular pace. The coverage of an election often loses the signal, and in fact is frequently busy accentuating the noise. In between the lines of Silver’s text is a recognition that news coverage affects polling numbers; in other words, news coverage affects elections. Noone knows exactly how yet, in part because of this signal and noise confusion referenced by Silver.
The 4th Estate Project would put forth that the next stage of development in predicting elections will be in deconstructing how polls are shifting as a function of the population’s exposure to the media and news coverage. As Silver writes and as is commonly lamented by PEW and ourselves here at the 4th Estate project, there is an abundance of noise in election news coverage. However, it is also clear that somewhere in the new coverage is a signal that is critical in shaping various populations’ perceptions of candidates running for political office. Exploring the nature of this signal, and understanding how this signal emerges from the noise that is embedded in most election news coverage is the work of the 4th Estate project. In essence, the 4th Estate Project is focused on algorithms that explore the relationship between news coverage and polls. In the future, if we are successful in our work, we will be able to predict elections – not from the moment when polls are published, but from the moment when information is published in the media; from the moment when information is distributed to the population via the media.