Bad Science: Quacks, Hacks, and Big Pharma Flacks

Nothing is certain. That’s why we use statistics– the analysis of numbers and data – to determine something’s probability, such as the effectiveness of a treatment or the likelihood that certain crimes are going to happen. When used correctly, they can be incredibly useful.

For example, statistics can be used inmeta-analysis, in which the results from many similar studies with few patients are combined into a larger, and therefore more robust and accurate test of whether a treatment is effective.

For example, between 1972 and 1981, seven trials were conducted to test whether steroids reduced the rate of infant mortality in premature births, and each showed no strong evidence to support their hypothesis.

However, in 1989 the results were combined and analyzed through meta-analysis, which found very strong evidence that steroids did in fact reduce the risk of infant mortality in premature births!

So wherein lies the discrepancy? The patterns in small studies are sometimes only visible when the data is aggregated.

Yet for all their worth, statistics can be misunderstood and misused, leading to bogus evidence and even injustice.

For example, a solicitor named Sally Clark had two babies who both died suddenly at different times. She was then charged with their murder and sent to jail because of the statistical improbability that two babies in the same family could die of Sudden Infant Death Syndrome (SIDS).

In fact, one key piece of evidence against her was the prosecutor’s calculation that there was only a “one in 73 million” chance that both deaths could be attributed to SIDS. However, this analysis overlooked environmental and genetic factors, which suggest that if one child dies from SIDS, the chances of another SIDS-related death in the family are more likely.

Not only that, but the chance that Clark committed double murder was actually twice as unlikely as both her children dying of SIDS, which, when considered with the rest of the evidence, meant that statistics themselves were simply not enough to convict her.