Right on Science?

Tech Central Station ^ | 05/25/2004 | Kenneth Green

[farmfriend]

Right on Science?

By Kenneth Green

"Papers are getting it right on science," crowed the Globe and Mail, citing the results of a study published in the Canadian Medical Association Journal (CMAJ) (Picard, 2004). But even in covering a study about media accuracy, the Globe and Mail got the story wrong -- What the CMAJ study actually showed was that more than one-third of newspaper stories about scientific studies held exaggerated claims, while nearly one-fifth held scientific and technical inaccuracies (Bubela and Caulfield, 2004).

In an April 27 article, "Papers are getting it right on science," André Picard wrote that "Contrary to the widely held belief among scientists and members of the public that stories are twisted and hyped to sell more newspapers, the reporting of scientific research in daily papers is actually pretty accurate, according to a new study."

The author of the study, Dr. Timothy Caulfield of the University of Alberta is quoted as saying, "Our data suggest that most newspaper articles accurately convey the results of and reflect the claims made in scientific journal articles."

In a CMAJ commentary regarding the Caulfield study, Celeste Condit writes that the results of the Caulfield study "…indicate that media reports are reasonably accurate, except in specific types of controversial areas, and that cases of inaccuracy may be as much a product of the researcher's over enthusiasm as of error by the reporter." (Condit, 2004)

But digging into the study itself suggests that the Globe article's headline, and optimistic lead paragraphs are a poor reflection of the underlying research, as are the comments characterizing the Caulfield study expressed in the CMAJ commentary.

In the Bubela and Caulfield study, the authors examined the reporting of papers in Canada, the US, Great Britain, and Australia on certain discoveries in genetics. Examining 627 newspaper articles reporting on 111 papers published in 24 scientific or medical journals, the authors found:

• 11% of newspaper articles had moderate or highly exaggerated claims;
• An additional 26%of newspaper articles had mildly exaggerated claims;
• 18% of articles had between one and three significant technical or scientific errors;
• Although only 41% of research used humans as research subjects, 87% of the scientific papers, and 98% of newspaper reports extrapolated the results to human beings; and
• Only 15% of newspaper articles examined made any mention of costs or risks, focusing instead on only the benefits of a given finding.

It is hard to see how getting one of every three stories wrong (whether errors of fact or errors of biased coverage) can possibly lead to the conclusion that "Papers are getting it right." Nearly one in five newspaper articles covering studies that are already selected, one presumes, because they're considered to have important health implications are scientifically or technically erroneous. One wonders if this level of diagnostic failure in medicine would be considered "getting it right," it certainly wouldn't at a grocery check-out stand, or at the bank, or when the dentist started drilling at a tooth.

In her CMAJ Commentary, Celeste Condit tries to cut the media some slack, emphasizing that the research itself, or the researchers when interviewed, may tend to exaggerate the importance of the work and the significance of its findings, and that may be the case. It's entirely possible that researchers will exaggerate the importance or significance of a given study for a variety of reasons. But one must ask, is it not the job of the science writer to be skeptical, and to seek out independent commentary from those who may be more dubious about the importance of a given study?

Reports in the media constantly raise alarms about scientific, medical, and technological issues. Various groups, from government to non-governmental activists, often propose regulatory activity with significant economic impacts based on such alarms. Whether it's reporting about genetics, as it was in the CMAJ study, or whether it's about reporting on pesticides, climate change, childhood asthma, or any of a hundred other risk issues that fill our newspapers, a 37% exaggeration rate and an 18% error rate is anything but "getting it right on science."

Kenneth Green directs the Centre for Studies in Risk, Regulation, and Environment at The Fraser Institute.

[farmfriend]

Nothing to see, move along.

[LiteKeeper]

Junkscience Reporting

[Ernest_at_the_Beach]

Reports in the media constantly raise alarms about scientific, medical, and technological issues. Various groups, from government to non-governmental activists, often propose regulatory activity with significant economic impacts based on such alarms. Whether it's reporting about genetics, as it was in the CMAJ study, or whether it's about reporting on pesticides, climate change, childhood asthma, or any of a hundred other risk issues that fill our newspapers, a 37% exaggeration rate and an 18% error rate is anything but "getting it right on science."

Right On!

[Ken H]

This is a gem. He really nails it in the concluding paragraph.

Reports in the media constantly raise alarms about scientific, medical, and technological issues. Various groups, from government to non-governmental activists, often propose regulatory activity with significant economic impacts based on such alarms. Whether it's reporting about genetics, as it was in the CMAJ study, or whether it's about reporting on pesticides, climate change, childhood asthma, or any of a hundred other risk issues that fill our newspapers, a 37% exaggeration rate and an 18% error rate is anything but "getting it right on science."

It is also important to consider the possibility that there could be a bias on the part of the news writers or researchers. The studies touted by gun control organizations come to mind.

You should always look to see who is funding the research.

[E.G.C.]

BTTT!!!!!!