A pollster critiques Nate Silver

Survey Says
By DAVID SCHARFENBERG  |  December 12, 2012

Professional pundits, conservative blowhards, and the math-challenged — sometimes one-in-the-same — went after New York Times blogger Nate Silver hard during the presidential race.

Silver, of course, engaged in high-profile aggregation of public opinion polls and declared President Obama the heavy favorite in a race his critics insisted was too close to call.

Election night brought vindication for the blogger — he called all 50 states correctly — and embarrassment for his prime antagonists. But it turns out there's an intelligent critique to be made of Silver and other aggregators like Simon Jackman of the Huffington Post.

Michael Dimock, a pollster and associate director for research at the Pew Center for the People & the Press, sketched the outlines of that critique during an appearance at Brown University's Taubman Center for Public Policy & American Institutions this week.

I caught up with him by phone the next day and asked him to flesh it out. The interview is edited and condensed.

WHAT IS THE APPEAL OF SILVER'S FIVETHIRTYEIGHT BLOG? Well, whenever there's a lot of data out there, anything that can help simplify it is appealing. It's why we like to rank college football teams.

I'm very sympathetic to the frustration a lot of people feel when it seems that polls might conflict, where they hear one poll saying Obama's up by four and another poll saying it's even. I understand the frustration people have there and I think the aggregators kind of smooth it out for people.

SO WHAT ARE YOUR CONCERNS WITH AGGREGATION? From a pollster's perspective, the frustration is that it gives a false sense of precision. The aggregations tend to move up by decimal points or down by decimal points on a daily basis and there's a tendency to want to read a message [in] those kinds of movements that's probably inappropriate, given that polls have margins of error and really aren't designed to deliver that kind of accuracy.

The second and larger frustration is that they focus all of the attention on the horse race of the poll and not the broader substance of the poll that is designed to explain why the candidates are doing better or worse, to explain what the voters' priorities are and how those priorities are linked to their choices, to uncover the voters' concerns and worries, to study whether this is an easy choice for people or a difficult choice for people.

Because aggregation is a simplification of polls, it ends up distilling all the focus to who's ahead and who's behind. And when you talk to people about what frustrates them about polls, their criticism is that that's all polls do — "oh, they're just telling us who's ahead and who's behind" — but we're not, it's actually the aggregators that are doing that with our polls.

AS YOU SAY, POLLS CAN POINT TO SOME OF THE "WHYS." YESTERDAY, YOU POSED A COUPLE OF QUESTIONS THEY MIGHT HAVE ANSWERED DURING THE CAMPAIGN: WHY IS OBAMA'S SUPPORT HOLDING UP IN A POOR ECONOMY? WHY IS MITT ROMNEY HAVING TROUBLE REPAIRING THE GOP BRAND? SO, WERE THERE SOME IMPORTANT "WHYS" THE POLLS UNCOVERED THAT WERE OBSCURED BY AGGREGATION? I think to some extent, yeah. And most of them were related to the economic issues that were at the forefront of the election. When you go through some of the particular economic policy debates, the public very much sided with Obama. They wanted to see taxes go up for higher-income Americans, they were insistent that taxes be at least a part of the conversation when we talk about deficit reduction.

1  |  2  |   next >
Related: Tyme fer moore lernin’, An angry electorate? Eh, whatever, Reader reactions to the Portland Phoenix endorsements, More more >
  Topics: This Just In , Nate Silver, Polls, statistics
| More

Most Popular
Share this entry with Delicious
  •   LIBERAL WARRIOR  |  April 10, 2013
    When it comes to his signature issues — climate change, campaign finance reform, tax fairness — Whitehouse makes little secret of his approach: marshal the facts, hammer the Republicans, and embarrass them into action.
    A key Brown University oversight committee has voted to recommend the school divest from coal, delivering a significant victory to student climate change activists.
  •   HACKING POLITICS: A GUIDE  |  April 03, 2013
    Last year, the Internet briefly upended everything we know about American politics.
  •   BREAK ON THROUGH  |  March 28, 2013
    When I spoke with Treasurer Gina Raimondo this week, I opened with the obligatory question about whether she'll run for governor. "I'm seriously considering it," she said. "But I think as you know — we've talked about it before — I have little kids: a six-year-old, an eight-year-old. I'm a mother. It's a big deal."
  •   THE LIBERAL CASE FOR GUNS  |  March 27, 2013
    The school massacre in Newtown, Connecticut spurred hope not just for sensible gun regulation, but for a more nuanced discussion of America's gun culture. Neither wish has been realized.

 See all articles by: DAVID SCHARFENBERG