Scrutinizing science behind the scenes

Science News,  June 21, 2008  by Tom Siegfried

• E-mail
• Print
• Link

Every issue, Science News delivers reports from the front lines of science--the latest findings from scientific journals, accounts of presentations at conferences, descriptions of events such as the Phoenix Lander's arrival on Mars. But reporting the news isn't the only task of science journalism. To put the news in perspective and context, it's important to go behind the scenes.

In this issue, freelance writer Regina Nuzzo explores the backstage machinations underlying much of today's biomedical science news--the mathematical methods used for analyzing studies of genes and disease. Over the past couple of decades or so, thousands of scientific papers have been published linking human diseases or other maladies to variants of specific human genes. In the preponderance of cases, those links turn out to be false. It seems that, very often, the math is misleading.

That is not very surprising to anyone who truly understands the ins and outs of probability and statistics, the branches of mathematics researchers rely on to draw inferences from complicated data. There is never any absolute guarantee that a statistical inference will turn out to be correct--just a likelihood. And standard methods are really not very good at quantifying that likelihood. Often statistical methods are improperly applied, and even when the math is done correctly the results are frequently misinterpreted, even by the scientists themselves.

In her article, Regina (herself a trained statistician) explores a relatively new approach that attempts to rectify some of the problems with previous studies linking genes to diseases. These "genome-wide association" studies employ a multistep process for paring down the massive amounts of data produced by genome studies, building in internal replication to eliminate (well, reduce) the prospect of false links.

It's certainly an advance over previous methods, which were overmatched by the complexity and massiveness of genomic data. But even the whole-genome approach has its limits and pitfalls. Our report on it should serve not just as a look behind the scenes at the methods that produce the news, but also as a reminder that science news is often flawed because it describes an imperfect process--science. Shortcomings that reduce the reliability of science's methods (and hence the news coverage of it) will be a topic for future discussion in these pages. It is the case, after all, that how scientists find out what they find out is sometimes as important as what they (say they) find out.

--Tom Siegfried, Editor in Chief

COPYRIGHT 2008 Science Service, Inc.
COPYRIGHT 2008 Gale, Cengage Learning