Scientists exposed as sloppy reporters

New Scientist ^ | 09:30 14 December 02 | Hazel Muir

[vannrox]

Scientists exposed as sloppy reporters

09:30 14 December 02

Hazel Muir

A cunning statistical study has exposed scientists as sloppy reporters. When they write up their work and cite other people's papers, most do not bother to read the original.

The discovery was made by Mikhail Simkin and Vwani Roychowdhury of the University of California, Los Angeles, who study the way information spreads around different kinds of networks.

They noticed in a citation database that misprints in references are fairly common, and that a lot of the mistakes are identical. This suggests that many scientists take short cuts, simply copying a reference from someone else's paper rather than reading the original source.

To find out how common this is, Simkin and Roychowdhury looked at citation data for a famous 1973 paper on the structure of two-dimensional crystals. They found it had been cited in other papers 4300 times, with 196 citations containing misprints in the volume, page or year. But despite the fact that a billion different versions of erroneous reference are possible, they counted only 45. The most popular mistake appeared 78 times.

The pattern suggests that 45 scientists, who might well have read the paper, made an error when they cited it. Then 151 others copied their misprints without reading the original. So for at least 77 per cent of the 196 misprinted citations, no one read the paper.

Spread like weeds

Still, you might think that the scientists who cited the paper correctly had been more dutiful about reading it. Not so, say Simkin and Roychowdhury. They modelled the way misprints spread as each new citer finds a reference to the original source in any of the papers that already cite it.

The model shows that the distribution of misprinted citations of the 1973 paper could only have arisen if 78 per cent of all the citations, including the correct ones, were "cut and pasted" from a secondary source. Many of those who got it right were simply lucky.

The problem is not specific to this paper, the researchers say. Similar patterns of errors cropped up in a dozen other high-profile papers they studied. The trouble is that researchers trust other scientists to repeat the key message of a paper correctly. This means that when misconceptions take root, they spread like weeds.

Simkin and Roychowdhury promise that between them they read all the references listed in their own paper including a book by Sigmund Freud. Their advice to other scientists is "read before you cite".

09:30 14 December 02

Return to news story

  © Copyright Reed Business Information Ltd.

 

[vannrox]

Hum.

[Willie Green]

In academia, the phenomena has long been known as "Publish or Perish".

It's related to the widespread pursuit of "junk science" as well.
Academic "researchers" publish "scientific" studies that support predetermined outcomes merely for the purpose of securing grants that are awarded politically.

[lelio]

studies that support predetermined outcomes
Right on the money there. That's highlighted by the obvious mis-referencing of articles by the author of a new paper. He doesn't care what's in the original article, he has his own agenda to persue.

[Dan(9698)]

>>> for the purpose of securing grants that are awarded politically.

Why go to a lot of work. If the conclusion isn't PC they won't get any more grants. Correct has nothing to do with it.

[jlogajan]

I'm sure someone will say this is the final proof that evolution is a fraud and that therefore the universe was indeed created in seven days, 7000 years ago.

[RightWhale]

misprints in references are fairly common, and that a lot of the mistakes are identical

Happens in textbooks, too. Even math textbooks. It's not hard to tell who has decided to write a textbook and who is on the forefront of math research. They aren't the same people. Those who write textbooks often come close to outright copying of examples and methods letter for letter. That, plus the mass of typos and poor grammar is either on the increase or we're just getting more aware of it.

[TEXOKIE]

Interesting post! Bump!

[LiteKeeper]

bump

[edsheppa]

It's possible to have read the original and to have copied an erroneous reference.

[185JHP]

Not to mention painting "tumors" on mice. Wunderkind in their own minds...

[Doctor Stochastic]

And how well do reporters report science?

[DBtoo]

So true. Many science textbooks have mistakes. A good professor will go through the book and find them and correct them so the students won't be mislead or confused.

There is a lot of sloppy science out there in the world of research, lots of cheating such as manipulation of data, and stealing of ideas from others goes on.

[VOA]

Many science textbooks have mistakes. A good professor will go through the book and
find them and correct them so the students won't be mislead or confused.

One biochemistry textbook I was burdened with during graduate school was rife with errors.
Given the name of the author, we came up with this phrase:
"It's Right or It's Rawn".
(yeah, the author's last name was Rawn)

There is a lot of sloppy science out there in the world of research, lots of cheating
such as manipulation of data, and stealing of ideas from others goes on.

As the venerable chairman of the graduate department said to me "about one-third
of published works are true and correct, about one-third are usable but have
relatively inconsequential errors, often in the form of honest mistakes; the final third
are full of errors and never should have been published."

I don't know if the breakdown is that even, but I do think that the external review
process used in the science world of the USA (and most of Western Europe) does a
good job of winnowing out the junk, while not overly constraining the flow of
new data/information.

But like any human system it does break down at times.
What is wrong is that more science frauds don't go to jail for mis-use of
government funds (which prop up about every research lab) when a clear
case of fraud is uncovered.

But, in a society where folks in many other institutions don't get the punishment
they manifestly deserve...it's no suprise that fraudulent scientists don't
often end up in orange jumpsuits.

[Darksheare]

Like the 'professor' who wrote that anti-gun paper who gto hoist by his own petard.. and then some?
Naaah. Scientists wouldn't do that! There's no politics in science! / sarcasm

[dark_lord]

...the universe was indeed created in seven days, 7000 years ago.

No, that was just the last "system reboot" after a serious local galactic arm system crash. Took that long to recover and reapply all the data. Which is why while day one was the "Let there be Light (1:3-5)", it wasn't until day 4 that the creation of the "Sun, moon, and stars occurred (1:14-19)". In a system recovery, you don't have to recover everything in the original order, but you do have to turn on the lights in the computer room while you are working.

Oh, what caused the local galactic arm system crash?

Just kidding. Maybe. :-)

[Alamo-Girl]

Thank you so much for the heads up!

[That Subliminal Kid]

You're as predictable as the sun.

[tallhappy]

This suggests that many scientists take short cuts, simply copying a reference from someone else's paper rather than reading the original source.

That doesn't mean they didn't read it. This is a false dichotomy.

New Scientist is a strange magazine.

[TopQuark]

This must be one of the most stupid studies on record: the assumption, that a scientist must read the original paper, is wrong. Since the study is done by information "scientists" (librarians), they simply do not know that.

When citing a paper, one often gives credit to the original inventor or developer of a result or an idea.

When someone refers to Newton's law, he almost certainly have never read Newton. I certainly hope not: this would be a compelte waste of time when studying science (as oppostite to history thereof).

[TopQuark]

Sounds like you've never even been to the same room with a scientist. But go ahead: demeaning someone make you look taller. The other way, of course, is to grow, but that's much harder: you'll have to know what you are talking about.