22 January, 2007

statistical analyses can be misleading

In another attempt to make matters seem worse than they truly are, USA Today posted a brief article on how "Americans" feel about the political situations currently on the table. I put "Americans" in quotes (again) for one simple reason: the poll presents itself as being representative of all Americans, when in reality it is only a very small percentage of the entire population. (The side note in the left margin of the above mentioned article shows that only 1,005 adults were polled in this survey).

I believe that what the writer(s) of these kinds of articles are banking on is that readers (unlike me) will simply focus on the bold print and ignore the small boring print. If such is this case, then the article appears to be saying something completely different--i.e., two-thirds (an ironic number, oddly enough) of this entire country are not optimistic about how things are going. If that were truly the case, then the 200,778,326* people expressing their dissatisfaction is a significant voice--and one that should be heard with great concern. But, only 1,005 people were polled, and two-thirds of that number is 670.335 (obviously the ".335" can be ignored), which translates into only about 1/450,000 of the total population.** Not nearly as impressive.

The language/rhetoric used in such articles can be misleading if one is not careful. Throughout the article, in conjunction with the notion of "two-thirds", the writer(s) use other terms to further the illusion--e.g., "most", "Americans", "the nation", "public", etc. This is not to suggest that the 1,005 polled are not American or not a part of this nation or not a part of the public; it is only suggesting that these 670 only represent 670 positions of the 301,016,980 people within this country. This is also not suggesting that these 670 positions are insignificant and should therefore be ignored simply because they are few; it is only suggesting that they not be considered definitively representative of the entire country. If the writer(s) of USA Today want to show that two-thirds of this country are dissatisfied with the status of various political issues, then survey the entire population of this country and see what happens.***

Statistical analyses, if not seen in their proper context, can be misleading. It is crucial for us readers of such polls not to overreact and/or believe the (false) reality being presented by such analyses. When an article, such as the one in question, says something like "Two-thirds think country headed down wrong track", we must see what that number actually represents instead of believing that actually two-thirds of this country hold that position. It would easy for us to read such a headline an be sucked into the conclusion: "Wow, if that many people are dissatisfied with the President and his agenda; then he must be doing something wrong"; when in reality, it is only 670 people who are dissatisfied. Therefore, the above conclusion would be a bit hasty to make. So don't make it, and realize why it would be bad to make it.

________________________________

* This figure is based on the population of America being 301,016,980, as of 11:30am on 22-Jan-07--according to the US and World Population Clock. It is understood that this number would be smaller given other key variables (noted below).
** Granted, there are a number of variables to consider--i.e., age, race, gender, social status, geographical location, political ties, etc.
*** Well, obviously survey those who are able to respond to such surveys.

1 comment:

George said...

1 out of 1 engineers agree that statistics can be presented in such a way as to "prove" either side of an argument. However, when studied in an unbiased manner (unlike the press presentation), even such a small sample size can give suprisingly good insights. The problem may not lie so much in the data as in the delivery.
I suggest that you personally begin a survey of each voter in the US to get us some truly meaningful numbers.