What Was Nate Silver’s Data Revolution?

After Donald Trump’s improbable victory over Hillary Clinton, in 2016, a lot of the same liberal readers who had once relied upon Silver as a comfort doll directed their anger toward all the quants who had given Clinton a sizable edge in the election. During the primaries, FiveThirtyEight was consistently dismissive of Trump’s chances of winning the nomination, but in the general election against Hillary Clinton, Silver gave Trump the best chances of any reputable election-forecasting outfit, at twenty-nine per cent. That twenty-nine-per-cent outcome was realized, as will be true twenty-nine per cent of the time. But Silver’s critics still felt like they had been misled—the oracle had seemed to have been debunked.

The number crunchers haven’t really recovered since. Poll analytics and data-driven political journalism haven’t disappeared in the past seven years, but the hope for an empirical alternative to Beltway reportage has certainly faded. But I don’t think the cooling of the public’s attachment to their favorite poll wizards had much to do with who got it right and who got it wrong. If that were true, Helmut Norpoth, a professor at SUNY Stony Brook whose model correctly called the election for Trump, would have become the next Nate Silver. Instead, it became clear that the gravity of the 2016 election and the real fear that the same liberals felt for the future of the country could no longer be processed by turning elections into a gambling proposition.

The question still remains: How do you gauge, say, Ron DeSantis’s chances to win the Republican primary? It’s obvious that the old-school political journalism that the quants rightfully tried to replace, with its cast of hustling aides and shadowy kingmakers, hasn’t improved. Just last November, that same machine was putting out stories about the incoming “red wave” in both the House and the Senate; reporters were pontificating about how John Fetterman’s poor performance in a debate against Mehmet Oz, which seemed to show the effects of his recent stroke, would doom him on Election Night.

The quants were supposed to provide a pathway out of the thicket of bad narratives, but that hope came from a fundamental misunderstanding of what the models were trying to measure. Silver was imagined to embody certainty when, in fact, his actual job was to inform the public of probabilities. “It’s very weird to become very well known for the wrong reasons,” Silver said. “People say, ‘Oh you have numbers and therefore a lot of certainty’ and they can’t quite process the fact that you can use numbers to quantify uncertainty as well.”

Silver, for his part, doesn’t quite seem to know what to do next. He is working on a book about gambling, and hopes his next media venture won’t be narrowly focussed on politics, in part because he doesn’t feel particularly invested in them. “I am a fan of the N.B.A.,” he said. “If there’s some palace intrigue about some coach, I’m listening to Bill Simmons’s or Zach Lowe’s podcasts. Or, if there’s drama in the poker world, I’m paying attention because I’m a fan of that world. Whereas, with the politics stuff, I just like the elections part.”

Split Ticket, a political-modelling Web site run by a group of twentysomethings, argues that there is a way to explain electoral politics to the broader public without falling into the oracle business. The site, which touts its ability to communicate numbers to the broader public, began during the pandemic, when Lakshya Jain, Harrison Lavelle, and Armin Thomas were talking on Twitter about polling and the upcoming 2020 election. Jain did not have a particularly distinguished start to his political prognosticating career. He had just finished a master’s program in machine learning and computer science at the University of California, Berkeley, and ran a calculation that predicted—disastrously—that Joe Biden would win four hundred and thirteen electoral votes. But, over time, Jain has improved his track record. In November, 2021 he, Lavelle, and Thomas launched Split Ticket, and were soon joined by Leon Sit and Clare Considine. Since then, they have nailed Georgia’s Senate runoff, and avoided the failed “red wave” narrative in the 2022 midterm elections.

As was true for FiveThirtyEight, Split Ticket’s most promising innovation comes from a permutation of sports analytics. Wins Above Replacement (WAR) gauges how many wins an athlete adds to a team relative to “replacement level,” which, roughly speaking, means the level of a bad player who would cost a team nothing to acquire. The hope is to provide a single number that conveys the full effect of a player’s impact on a game. Split Ticket’s version of WAR tries to do a similar thing with candidate quality by providing a “quantifiable ‘score’ for each district that displays whether the Republican or Democrat performed better relative to data-based expectations.” Just as baseball analytics adjusts for ballpark effects or the quality of a team’s fielding, Split Ticket controls for factors like the “racial composition” of a seat, incumbency, over-all voting trends within the state where the election took place, and the money that each candidate spent. A candidate, then, may be swept into office as part of trends up the ballot, but WAR attempts to isolate the candidate’s individual quality from those external factors.

What Jain and Split Ticket hope to bring to the political conversation is a retrospective look at political outcomes which cuts against the typical postmortems from the commentariat. Jain talked about pundits who blame every Democrat election loss on the candidate’s flirtations with defunding the police in the summer of 2020. “When people talk about candidate quality in the national media through a more holistic lens, they subconsciously project their own biases onto the results and try to explain it in ways that may not necessarily comport with what the reality actually was,” Jain said. A metric like WAR, then, tries to disambiguate and detach a candidate from the convenient existing narratives.

Their formula found, for example, that even though the Democrat Charles Booker lost his Kentucky Senate election against the Republican incumbent Rand Paul, Booker actually outperformed his expectations by 8.1 per cent. The Wisconsin race between the Democrat Mandela Barnes and Ron Johnson, another Republican incumbent, included a great deal of speculation that what the Wall Street Journal called Barnes’s “progressive ties” could ultimately hurt his electability. Split Ticket found that, though Barnes did ultimately lose to Johnson, he still outperformed the theoretical replacement-level candidate by four points.

Before Amy Klobuchar ran for President, much was made of the strength she had shown in her senatorial runs, particularly in rural parts of Minnesota. The case for her candidacy was cast in quantitative terms—she was a Democrat who won her 2018 Senate race by a twenty-four-point margin in a state that Hillary Clinton had won by only two points in 2016, and she held an unusual edge among white, non-college-educated voters. Some pundits imagined that Klobuchar had special qualities that would translate to wins in other swingy Midwestern states such as Michigan and Wisconsin. But by controlling for the gap in campaign spending between Klobuchar and her opponent, as well as Klobuchar’s incumbency advantage, Split Ticket found that she had outperformed expectations only by six points—still impressive, but certainly not anything to build a Presidential campaign on.

WAR is by no means perfect. Like any model, its quality will ultimately depend on how well the Split Ticket team can adjust its controls and keep pace with a complex, changing voting public. And Jain and his colleagues can’t control how their numbers might get swept up in partisan narratives. In sports, these types of all-in-one metrics tend to gain traction with the general public when they flatter the consensus of their audience. Confirmation bias, buoyed by the political desires of one’s audience, becomes the arbiter of whether a given metric is seen as good or bad. It’s likely that Split Ticket’s success will depend on the same impulses that made people misread Silver for so many years.

But that doesn’t mean the quants aren’t doing something valuable. Our industry still has not extricated itself from the Beltway-gossip model in which a writer talks to a few people in D.C., puts his finger up to the wind, and takes what amounts to an educated guess. The truth, however unsatisfying, is that, though the polling wizards are fallible, invite a type of misreading, and perhaps do not always live up to the gravity of the moment, they’re still better than what came before. ♦