Of course, everything turns on the accuracy of the surveys. Most of us, not being statisticians, more or less take them on faith. If Wuthnow is right, though, our faith is misguided. He points out that many surveys of American religion have serious methodological flaws. For example, religion does not always lend itself to straightforward yes/no questions of the sort surveyors ask. In addition, pollsters sometimes fail to account for regional and racial variations.
Most important, response rates are very low. The typical response rate nowadays is about nine or 10 percent, and rarely exceeds 15 percent. “In other words,” Wuthnow writes, “upwards of 90 percent of the people who should have been included in a poll for it to be nationally representative are missing. They were either unreachable or refused to participate.” With such poor response rates, it’s hard to know what the polls reveal about religion in America. This problem is compounded by the fact that the media present the results as accurate representations of what Americans believe””a misimpression that the polling industry, now worth a billion dollars a year, is understandably reluctant to correct””and by the fact that most of us “are unlikely to wade through obscure methodological appendices to learn if the response rate was respectable or not.”
Read it all.
(First Things) Robert Wuthnow Discusses Some Serious Flaws in polling about US Religion
Of course, everything turns on the accuracy of the surveys. Most of us, not being statisticians, more or less take them on faith. If Wuthnow is right, though, our faith is misguided. He points out that many surveys of American religion have serious methodological flaws. For example, religion does not always lend itself to straightforward yes/no questions of the sort surveyors ask. In addition, pollsters sometimes fail to account for regional and racial variations.
Most important, response rates are very low. The typical response rate nowadays is about nine or 10 percent, and rarely exceeds 15 percent. “In other words,” Wuthnow writes, “upwards of 90 percent of the people who should have been included in a poll for it to be nationally representative are missing. They were either unreachable or refused to participate.” With such poor response rates, it’s hard to know what the polls reveal about religion in America. This problem is compounded by the fact that the media present the results as accurate representations of what Americans believe””a misimpression that the polling industry, now worth a billion dollars a year, is understandably reluctant to correct””and by the fact that most of us “are unlikely to wade through obscure methodological appendices to learn if the response rate was respectable or not.”
Read it all.