I promised to warn you about some of the ways you can be deliberately or accidentally misled by statistics, so you can guard against these tricks and so you can be careful not to inadvertently be guilty yourself. Here then is one popular misuse of statistics I see all the time.
This is to exaggerate the importance of a minor change in something or a small difference between two things. This is often done by journalists trying to create a story out of a factual report which does not actually reveal anything very exciting. To build it up into a story, the journalist usually leaves out a lot of the statistical information or presents it in such a way as to draw attention to the selected element only. Of course journalists are not alone in this respect. Sadly, we all too often find politicians, advertisers and salespeople doing the same thing.
There are three variations on this.
The first way is to report on a large percentage change, or difference, without pointing out the absolute size of the figures.
- If a new medical treatment or “healthy diet” reduces your chances of developing a certain disease by 25% it sounds worth trying, even if it costs a lot of money. Unless you know that your original chances of getting the disease were only 1 in 40 million, meaning the improved treatment or diet reduced it to 1 in 30 million. So what?
- If a certain type of crime has increased by 50% it sounds as if something must be done and soon. Unless you know that the average number for several years was 10 crimes a year, and last year there were 15. Is this something to worry about?
The second way is almost the opposite. It is to emphasise the absolute numbers without mentioning the percentages.
- If 100 people have complained about a product it sounds pretty bad. Unless you know that 100,000 people bought the product last year, so only 0.1% had anything to complain about.
- If an additional 20 people were killed in industrial accidents last year, you might think we were losing our obsession with Health & Safety, if you did not know that the total number killed in any year was several hundred.
The third way is to fail to mention the size of “normal” fluctuations in whatever is being measured.
- If the number of motor accidents has gone up 10% in the last 3 months it sounds like a recent road safety campaign was a failure. Unless you know that the number varies by up to 15% from one month to the next almost all the time.
- Two very severe winters in consecutive years can sound like the start of a new Ice Age. Until you look at the fluctuations in temperatures between years over a long period and see that we get bad ones every decade or so.
So always look at both the percentage differences and the absolute numbers before deciding it is time to shout “don’t panic!” or to demand “action”.
[N.B. All figures stated above are made up for purely illustrative purposes.]