The problem with polls, and the media (including the blogosphere.)

Polling can always tell us whatever the person who constructs/conducts the poll was investigating - if we're given the raw data and a good description of the sampling procedure. But in practice even the data is usually glossed over in favor of a sound-bite summary tending to support the interests of the person and/or network doing the reporting on it.

Unless you know about how the sample of people was selected you really can't know anything more than what's reported about a poll. You can't know, for instance, if its findings are useful in any logical sense, because you don't know who the sample represents.  I can ask 21 people a question, and come back with really convincing looking numbers, but if I select who 15-20 of those people are it will darn sure tell you what I want you to think I learned.

An example of shaping a poll

Imagine I go to a GOP Town Hall meeting, and survey 15 people wearing shirts or carrying signs that say either "Nobama," or, "Joe Wilson was right!" I'll ask them one simple question:

Are you a) "for" Obama's government takeover of our health care system that he's pushing through the congress under the name of "reform" or b) "against reform" that will make changes that undermine the free market system that has given us the best health care in the world and cost the tax payers even more money?

OK, I've plausibly got 15 "b) against reform" responses now in my hypothetical example.  I'll ask 6 additional people, more or less randomly selected, and let's say they most of them magically favor reform (not likely, is it? But for the sake of argument, I'm getting 4 out of 6 favorable replies.)  I didn't even tack on the line about paying for illegal immigrants.

Now I'll report back for you based on that (fake) survey:

"In a [hypothetical] survey conducted Wednesday, only 19% of those responding favor the proposed reforms to health care, while  nearly 81% said they were 'against change.' That's more than 4 out of 5 in our survey who are hoping their representatives in Congress will stop the President's take-over of business."

If you believe what anybody in the media tells you without understanding both the sample and the data, all you know is what the reporter's boss wants you to believe. If you choose to believe on that basis - which you just might if it agrees with your political leanings - rather than examining the poll itself, then you're gullible indeed.  The good news is: the politicians on your side and the ratings-hungry networks (who are on the side of earning a living from ad revenues) both love you. They'll go out of their way to validate your "wisdom and insight" into the issue.

If the poll isn't conducted on a random sample, but merely open to those who respond...? Well, my friends, that will tell you a bit about the people who responded, of course, but one must be wary of extrapolating to draw any useful conclusions about a larger population. We call it spin. But knowing that they're gaming us doesn't stop the echoes.

How the media deliberately spreads misinformation

In fact, it won't surprise me to find this utterly fake survey example quoted elsewhere within days, if not hours.  Can't you see it, at DIGG maybe, or on another blog, or even on Fox?

A post at a prominent, liberal-leaning blog on Wednesday described a survey which concluded that, quote, "only 19% of those responding favor the proposed reforms to health care, while nearly 81% said they were 'against change.'" In other words, that's more than 4 out of 5 who want their representatives in Congress to stop the President's assault on insurance providers and let capitalism work.  

There you go, it's been lifted carefully out of context, and the quote is nearly character for character what I made up in the "report" above, and then the media echoes will persist even though the numbers are clearly unreal.  You see, now they're not reporting on the survey, they're reporting on the reporting, which is just an excuse to keep repeating the misleading numbers.

Misinformation mars the debate. I could easily have made the example go the opposite way, of course, but I don't want somebody to echo a story that falsely represents support for reform.  In fact, worded carefully surveys do reveal that over 90% favor "at least some reform."  But then, who wouldn't favor "at least some" unless they were making money from the insurance industry? It's like asking who wants lower taxes without considering how you'd pay for those government services you realize you benefit from.

You know that commercial media outlets rely on advertising revenues. So, do you follow the money? Better yet, why do you trust who you always have to report on things you care about?