Statistical (un)reality part 1

The old saying is that there are 3 types of lies; Lies, damn lies and statistics. The problem with statistics is not that they are necessarily untrue, but that they can be so easily manipulated to misrepresent what they actually represent. The ways that statistics can be misrepresented are legion, from the methodology with which the stats are collected to overreaching with the conclusions of the data. In the first of an irregular series on the misuses of statistics we will look at one example, treating popular opinion as statistically valid.

The causal links between video games and violent behavior have been championed by the media, typically by linking violent acts to a video game player. Some of the opinion pieces that get published seem to presume that the link is simply a matter of fact. Statistical representations like this
Source destructoid.com.
purport to show that a large proportion of violent game playing children are more prone to get into fights by representing the data in a graphic that has their bar as the largest. This is an accurate representation of the data, but not of the conclusion that can be drawn. While the number of fighting children that play video games is the largest segment of this sample, the video gaming group as a whole is much larger than the low violent game play group. Not representing this data proportionally suggests a false conclusion.

What that particular data shows is that 87% of low violent game play students get involved in fights while only 62% of violent game playing students do. It can also show that 90% of non violent kids play violent games but only 69% of violent children do. Both of these interpretations show that children that play violent video games are less violent in real life.

I found this graph on Digg the other day which was quite interesting. While it also uses a statistical representation trick to make the slope appear bigger (manipulation of the y-axis) it shows a trend in violent crime over the period that violent video games have been popular. While the downward trend is exagerated, there is no question that the data is real and it can be viewed at the Bureau of Justice source.

So here are my first tips on spotting dodgy statistics
- Beware of "conventional wisdom" expressed in statistical terms
- Look at the fairness of the comparison, somethings should be numerically compared, some should be proportionally
- Beware of the exaggerating effects that playing with scale can have.