Posted on 05/01/2006 9:18:06 PM PDT by neverdem
The Doctor's World
Recent disclosures of fraudulent or flawed studies in medical and scientific journals have called into question as never before the merits of their peer-review system.
The system is based on journals inviting independent experts to critique submitted manuscripts. The stated aim is to weed out sloppy and bad research, ensuring the integrity of the what it has published.
Because findings published in peer-reviewed journals affect patient care, public policy and the authors' academic promotions, journal editors contend that new scientific information should be published in a peer-reviewed journal before it is presented to doctors and the public.
That message, however, has created a widespread misimpression that passing peer review is the scientific equivalent of the Good Housekeeping seal of approval.
Virtually every major scientific and medical journal has been humbled recently by publishing findings that are later discredited. The flurry of episodes has led many people to ask why authors, editors and independent expert reviewers all failed to detect the problems before publication.
The publication process is complex. Many factors can allow error, even fraud, to slip through. They include economic pressures for journals to avoid investigating suspected errors; the desire to avoid displeasing the authors and the experts who review manuscripts; and the fear that angry scientists will withhold the manuscripts that are the lifeline of the journals, putting them out of business.By promoting the sanctity of peer review and using it to justify a number of their actions in recent years, journals have added to their enormous power.
The release of news about scientific and medical findings is among the most tightly managed in country. Journals control when the public learns about findings from taxpayer-supported research by setting dates when the research can be published. They also impose severe restrictions on what authors can say...
(Excerpt) Read more at nytimes.com ...
Well, very few journals ["Organic Syntheses" is a good example] publish the submissions only after the independent checkers [peer reviewers] have successfully replicated them. For science at large the duplication of all published research is cost-prohibitive.
And if anyone can speak authoritatively about credibility gaps....
Makes me glad I work for an honorable PhD. They are out there, and they are in the majority, at least in my research experience.
Peer review can't guarantee that the papers are correct. The best it can do is to pass judgment on whether the practices (as reported, and assuming good faith) were sound, whether the conclusions were supported by the data, and whether the disclosures were full enough for the work to be reproducible (or refutable, as the case may be). Obviously that leaves a wide gap between truth and publishability...but that's the way it ought to be, and indeed must be.
That said, I do view many medical papers with professional distaste. It seems to be a standard practice to claim medically significant effects based upon statistical cases that are so weak as to be consistent with no effect at all. These then get amplified by the uncritical press into medical certainties.
It would be bad enough if these suggestions-dressed-as-prophecies swayed only the impressionable public, but all too often the government and other powerful entities use them as the basis of policy decisions.
"Virtually every major scientific and medical journal has been humbled recently by publishing findings that are later discredited"
Some one doesn't understand how peer review works. :P
Peer review isn't just the guys who give the go-ahead on articles, but also the criticism of all of the scientists, "peers", who read the journals.
The bunk was spotted. The system worked.
Of course, I wouldn't be the first one to say that the intelligence of some medical doctors is a bit over rated.
Both JAMA and NEJM have been practicing "advocacy science" for years -- which bites you in the arse, peer-review or no.
The Hwang and Sudbø incidents are a little different -- the whole world of scientific publication is fundamentally an honour system. Here we have guys who broke the code. The editors, though, especially of NEJM, have been breaking the code themselves "in support of higher truths" for years.
Some of the stuff in that journal that purports to be epidemiological is worse than Hwang's misconduct.
d.o.l.
Criminal Number 18F
Caveat emptor, especially in medicine, but that attitude also has the FDA being attacked for being too tardy.
Be that as it may, I hope most folks understand the political correctness which apparently controls the various, premier English language journals, i.e.JAMA, NEJM, BMJ, the Lancet, Nature and Science.
I concur with your "honor system" observation. Scientists as a rule tend to understand what is needed to advance an agenda. If someone is going to break the rules and manufacture data, that person is also likely to come up with a "good" lie that will pass the first smell test. No different than practitioners of other occupations (media perhaps?) that can spin a good tale. The verification of the validity of the study comes in as others attempt to replicate the results.
New Findings Support Promise of Using Stem Cells to Treat Neurodegenerative Diseases
FReepmail me if you want on or off my health and science ping list.
But aren't scientists models of integrity because they are interested in only knowing the truth?
*Publish or perish* sounds like a corrupting factor.
Hmmmmm...lets see...a discredited "paper of record"...bit*&^ing and moaning about the credibility of scientific peer review....hmmmmm Ya think the NY Fish Wrap thinks we at Free Republic do "peer review" on journalists?!!!
This is all garbage - excuse making. I can't begin to count the number of times some elitist has made reference to how many times his cited source has been "published" in the peer review process. Nor can I count the number of times that the same types condescend toward subjects they don't like on the
basis that a paper on it has not been "published" and reviewed. The fact is, a set of idealogues control the process in order to ensure the outcomes. If you control what gets reviewed, you also control what doesn't. And you can manipulate the economic and educational market places with what does or does not. One is right to smell a rat. And it exist as much in this written excuse as with the process itself.
bookmark
Peer review is little more than mob rule. It is group-think control over life itself.
These actions are right out of my eighth grade 'social studies' text ('57 - '58) It's called propaganda; the swayed public opinion is used as justification for action by these agencies. Fallacy is the operative word.
Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.