Gallup's Frank Newport, Out of Ammunition and Beseiged in Silvergrad, Refuses to Surrender: War on Nate Silver Weblogging
Sam Wang turns up the pressure:
Gallup’s man misses the point: Gallup editor-in-chief Frank Newport appears to be on a campaign against poll aggregation…. Oh, please. Considering Gallup’s performance in estimating the national race, this could be interpreted as a defensive move by this year’s equivalent of the Literary Digest poll (‘Landon by a landslide,’ George Mason University). Newport misses the positive value that we bring to his activity. It is too bad, because what we do can ultimately increase the relevance of his organization. Here’s why.
First and foremost, poll aggregation is not like other forms of news aggregation. News aggregators like the Huffington Post basically recycle stories. Those of us who examine groups of polls add value…. For the reader, we cut through the noise. Individual polls contain two kinds of error arising from (a) inherent limitations of sampling, and (b) systematic errors made by individual pollsters. By using robust statistical tools, we reduce and cancel these errors to obtain a far superior result. For the pollster, we offer a benchmark for future performance…. [A]ggregators can grade a pollster’s accuracy to within 1 percentage point. Newport does not fully acknowledge the second point. Regarding his own organization’s performance he writes:
The “gap” difference was….well within the statistical margin of error and underscore[s] the accuracy of random sampling today.
Actually, no. Thanks to aggregation, we can say with great specificity that Gallup’s national October numbers (Romney ahead by 2% to 6%) were systematically off by 4-8% from the true margin at the time, Obama +2.0% (“A final unskewing,” November 12th). No wonder he doesn’t like us. Underneath the bluster, I believe that Newport’s real problem is his own organization’s performance….
[I]t was not only Gallup whose national numbers were off. National polls as a group were biased by an average of 2.4 +/- 0.4% toward Mitt Romney. State polls were a superior source of information…. Newport is correct that poll aggregation does devalue the news value of any single poll. That is the point…. I almost blew a blood vessel when I saw the entire front page of USA Today dedicated to a single Gallup poll that was an outlier. Let’s face it, news organizations love outliers. If aggregation kills that kind of story in the future, our entire nation wins….
[Pollsters] can’t get stuck in a rut reporting only topline numbers. That low-hanging fruit will soon be gone. But there are many ways they can improve their game. For example: Focus on crosstabs…. Watch one another….. Develop new products…. Learn from the crowd – but don’t be afraid to go the other way…. Go local!…
And here we come to an irony: the Gallup organization is rich in expertise, and is a leader in adding value in interesting ways. If they continue to do that – and stop complaining about the new kids on the block – they can maintain their relevance. I wish them every success.
