60 Minutes Gets Substandard C Grade for Medical Reporting in ACSH Study

The much-honored CBS program 60 Minutes, the highest rated public affairs program in television history, gets a substandard C grade in its reporting over the years on medical treatments and health threats, according to a detailed new study by the American Council on Science and Health.

60 Minutes has a reputation for credibility and its correspondents have received more than eighty major broadcasting awards, though specific 60 Minutes programs on health claims have been the subject of published criticism before (including those on shark cartilage as cancer treatment, toxicity of mercury amalgam fillings, various arthritis treatments, and Ajar as a cause of cancer). Despite this, the authors of the study say they are aware of no previous systematic evaluations addressing the accuracy of the program's reporting on health topics.

William M. London, Elizabeth M. Whelan, and Ruth Kava of the ACSH identified 135 segments on health and environmental topics aired from January 1978 through November 1995. They purchased all 128 transcripts that were available (seven were on legal hold). After eliminating those that dealt with issues of health care delivery and human interest profiles, they arranged for evaluations of 97 transcripts that addressed alleged health threats, health advice to consumers, and medical information.

Each transcript was reviewed by one or more panels, each consisting of three experts recruited as unpaid referees. There were a total of 66 referees, including 33 with M.D. degrees, 24 with PhDs, and the others with various other medical-related degrees such as DDS or PharmD. Each expert completed for each assigned transcript an evaluation form addressing issues of scientific accuracy. The form included eight items with five response options ranging from "strongly agree" to strongly disagree and one question asking for an overall "report card grade" ranging from "F" (failure) to "A" (excellent) that was recoded as integers from 0 to 4, and four questions eliciting comments.

The eight items used by the referees were four positive and four negative statements such as "the segment emphasized a scientific perspective," "airing the segment addressed a genuine priority for promoting public health," "all scientists or heath care experts interviewed as experts appeared to have relevant qualifications," "the segment sent an extreme rather than mainstream message," "the segment relied too much on anecdotal rather than scientific evidence," "the segment was deceptive," and so on.

The authors note that the study had limitations and advantages for quality assessments similar to those of the process of peer reviewing manuscripts submitted to medical journals. It's an imperfect process, they acknowledge. Nevertheless, they say, "Despite the limitations of peer review we believe our data probably provide a reliable indication of which stories on 60 Minutes were obviously flawed or invalid."

The top-ranked segment was the 1994 program "Less Than a Miracle," an investigation and expose of Facilitated Communication in the treatment of autistic children. It received a perfect 4.0 ranking from the referees. A 1990 segment on acid rain was next, at 3.8.

The adjusted mean of the grade point averages was 2.0 -- equivalent to a C grade. Less than 25 percent of the transcripts received mean grades greater than 3.0 (good) while more than 40 percent of the transcripts received mean grades less than 2.0 (fair).

"The scientific reviewers who performed the study cited clear misstatements or falsehoods in 46 out of the 97 segments reviewed," says London, the lead author. Notes co-author Elizabeth Whelan: "The 60 Minutes segments reviewed in the ACSH study ranged in quality from egregiously misleading to excellent, accurate reports."

Errors of inattention to evidence and of imprudence were common, say the authors. The most frequently cited problem was that the transcripts relied too much on anecdotal evidence. Reporting accuracy was judged only fair, on average.

The charts presented here, condensed from more detailed charts in the published study, list the 12 segments that received a better than 3.0 grade in the study (ranked 1 through 12) and the 17 segments that received a grade of less than 1.0 (ranked 81 through 97 in the study). The rank, segment name (topic), year, and mean grade are given.

The study identified the following common errors m 60 Minutes' reporting:

* Omission of a balancing viewpoint or scientific perspective (33 segments)

* Omission of relevant quantitative data (34)

* Sensationalism or hype (29)

* Errors of implication (29)

* Unsupported assertions (28)

* Overreliance on anecdote (21)

* Omissions of principles of data evaluation (19)

An example of a clear misstatement or falsehood identified in the study was suggested in this referee comment about the 1993 program "Sharks Don't Get Cancer": "Lied that sharks don't get cancer. In fact, they nor only get cancer, they get cartilage cancer." An example of "failure to provide proper context" was this referee comment on the 1990 segment "Is There Poison in Your Mouth?": "The segment failed to mention very important research supporting amalgam as safe. I know they had the research because I had sent it to them." An example of omission of a balancing viewpoint or scientific perspective" was this referee comment on a 1990 segment, "The Spraying of LA," ranked 97th--dead last--in the study: "Segment failed to indicate the low toxicity of malathion. Segment failed to indicate that malathion breaks down to nontoxic compounds very quickly after it has been sprayed."