I stared unseeingly at my television screen until four o’clock Wednesday morning, long after Donald Trump gave his very conventional and conciliatory victory speech and all the talking heads went off the air. Sure I’m worried about the future. But what left me cotton-mouthed and agog that night was how surprised I was at how surprised I was.
I cannot fathom how I could have failed to understand so many of my fellow Americans so badly. Of course I knew that Trump had millions of followers, but like a lot of us who pull on our smarty pants every morning, I felt quite sure that it was just a vocal and angry if better-than-usually-organized minority—you know, those hillbillies J.D. Vance elegizes, and some Rust Belt denizens too passive to extract themselves from the exhaust pipe of history. (And for some of our all-too-literal commenters, let me make explicit that this description is meant to mock and exaggerate attitudes that I think were lurking quietly in many predictions of yesterday’s election. My point here is that it’s foolish to judge what you don’t yet understand. Once you understand it, sure, judge all you like, and I intend to. But first you have to know and respect it as real.) To be sure, I had help from the mainstream media and an entire industry of professionals who make it their business to explicate and quantify the national mood. But I accept the error, and the humbling that fairly accompanies it, as my own.
What I really want to know is not so much how Trump won as how all us folks who pride ourselves on knowing what we’re talking about got it so wrong. These days, the pros not only give us a conventional margin of error (e.g., Hillary up by 4 plus or minus 2 points), but an outcome prediction with a probability or reliability assessment. For example, the New York Times said on Election Day that Hillary had an 85% chance of winning. Literally what that meant was that there was roughly a 1-in-6 chance they were wrong. Now maybe Trump's chances really were the odds of rolling a die once and getting a snake-eye. I doubt we'll ever know, but I’d bet that by election day a more effective assessment of the national mood would have suggested a much greater likelihood of Trump's winning, something a lot closer to even. Nate Silver more cautiously handicapped it at 70-30. Even that doesn't seem to capture it.
Because really it's not that much like predicting what a rolled die will show. It's an assessment of how confident you are in your methods. The pollsters and algorithmiacs systematically overestimated their confidence in their methods, or at least I’d bet they did. They didn't know what they didn't know, and couldn't quantify their uncertainty because they didn't know what they might be missing. The result was a surprise not because it was some random stroke of fate, like a winning lottery ticket, but because as hard as we looked at everything we just didn’t see some very pervasive and important stuff that was demonstrably there.
So what happened? What identifiable and measurable phenomena went unidentified and unmeasured? And why?