Free Republic
Browse · Search
Bloggers & Personal
Topics · Post Article

To: pepsionice
The key to fixing this entire mess...for all sides...is peer review.

Really? There was plenty of peer review going on in climatology: the AGW types reviewed papers that came to different conclusions and saw to it they were rejected, they reviewed each others' papers and accepted them for publication, fuzzy methodology, fudged data, etc. all overlooked so long as AGW was supported.

Peer review is broken: it creates herd-mentalities even when it isn't supporting actual corruption. Look at string theory--40 years w/o a testable prediction, 40 years of fudging to explain away the not-observed-in-nature dilaton field, but the sting theorist review each others' papers and grant proposals while competing ideas mostly get published in mathematics journals rather than physics journals.

The same thing happens in the social sciences. There was an experimental study done on social work journals: two fake research papers both with glaring methodological flaws were prepared, identical except that one purported to show a social work intervention worked, the other purported to show that it didn't. They were submitted to randomly chosen social work journals. Peer review accepted the social-work-works version more often than not, and rejected the social-work-didn't-work version more often than not. Of course, the deception was revealed and the submissions withdrawn, but the point was made.

Peer review depends on having a sufficient proportion of a discipline be committed to objectivity, and ideally not so committed to their own ideas that they can't see the merit in other, even competing, ideas. The 'big science' funding model used in most developed countries now militates against this, since money going to competing ideas isn't going to one's own ideas.

7 posted on 11/25/2009 10:41:38 PM PST by The_Reader_David (And when they behead your own people in the wars which are to come, then you will know. . .)
[ Post Reply | Private Reply | To 3 | View Replies ]


To: The_Reader_David
As someone who is in a "peer reviewed" discipline, and who has published in "peer reviewed" journals, it's often (not always) bogus.

*Almost everything depends on the editor of the journal and his/her prejudices. He/she determines WHO "peer reviews" the study. So if, say, "Nature," has a pro-GW editor, there is no way that study gets out, no matter how valuable or legitimate its research and findings are.

*Peers have their own axes to grind. I once sent in a paper (eventually published by the editor who overrode the reviewer) that was sent to the very guy I criticized. OF COURSE he found my conclusions "unconvincing." More often than not, even "blind" reviews can be figured out pretty easily: you know who is working in what fields, and from conferences, you know who thinks what.

*There is funding on the line. I don't think this is nearly as powerful a motivator as #1 and #2 above, but it is a motivator. If you're getting your $$ from the government, and a new study will threaten that, you're not likely to find much value in that study. As with lawyers and "expert witnesses," the expert witness NEVER comes to a conclusion that is antithetical to the lawyer's position (the guy who is paying him).

I might add that in 25 years in universities, I've never once seen a "pilot" program that resulted in the administration or department concluding, "You know what? That was a waste of time and we're not going to implement it." If it even GETS to "pilot" status, they will always conclude that "it works," if for no other reason than to justify the pilot funding!

Finally---and I'd like other academics to chime in here---there has been SOME evidence that scholarly studies/scientific studies can say one thing, and that the ABSTRACTS, occasionally written by editors, makes it sound like the study says something else. (Michael Crichton pointed this out in his book, "State of Fear"). Well, the idiot news media NEVER reads the studies, only the abstracts!

12 posted on 11/26/2009 5:06:03 AM PST by LS ("Castles made of sand, fall in the sea . . . eventually." (Hendrix))
[ Post Reply | Private Reply | To 7 | View Replies ]

Free Republic
Browse · Search
Bloggers & Personal
Topics · Post Article


FreeRepublic, LLC, PO BOX 9771, FRESNO, CA 93794
FreeRepublic.com is powered by software copyright 2000-2008 John Robinson