Free Republic
Browse · Search
News/Activism
Topics · Post Article

Skip to comments.

Most scientific papers are probably wrong
New Scientist ^ | 8/30/05 | Kurt Kleiner

Posted on 08/30/2005 10:29:44 AM PDT by LibWhacker

click here to read article


Navigation: use the links below to view more comments.
first previous 1-20 ... 81-100101-120121-140141-158 next last
To: adorno
...wouldn't the lattter paper be only 25% correct?

The claim is that a little more than 50% of all scientific papers are wrong (so that half of them are right, no?), not that each and every paper is 50% wrong in its contents. That WOULD be a major disaster, lol!

BTW, when I was a grad student in statistics we read an article in one of the stat journals that said that 80% of all the articles published in scientific journals have insufficient "power" (the probability you'll reject the null hypothesis if it's wrong). Talk about shooting yourself in the foot! Here the researcher spends years and years collecting data trying to demonstrate some earth-shaking effect, and yet he designs his own study to have only the slimmest chance of detecting the effect, EVEN IF IT EXISTS! And yet these papers still managed to get published, claiming they have enough evidence to reject the null hypothesis (the skeptic's hypothesis). That's telling us something, right there.

101 posted on 08/30/2005 12:58:00 PM PDT by LibWhacker
[ Post Reply | Private Reply | To 86 | View Replies]

To: talleyman

lol


102 posted on 08/30/2005 12:59:05 PM PDT by LibWhacker
[ Post Reply | Private Reply | To 95 | View Replies]

To: Cold Heart
Aw, come on. You're complaining about the notation? It's elementary statistics: I learned about sigma in public high school, decades ago. "Standard deviation" it's also called.

The publication standards are 3-sigma and 5-sigma, as I said. I spelled out their values (approximately) just because I realized that not everyone would know or remember what they meant. In all seriousness, I worried whether I was being too pedantic in explaining them.

103 posted on 08/30/2005 1:01:12 PM PDT by Physicist
[ Post Reply | Private Reply | To 91 | View Replies]

To: inquest
So you'd classify this as a relatively recent problem?

I've only been publishing and doing peer review for about 10 years now. The stat software has been available for that entire time. So I really don't have an opinion on whether it has changed. Part of the problem is that the peer-reviewers are, like the authors, usually specialists in the subject matter of the Journal and frequently are not strong in statistics.

104 posted on 08/30/2005 1:03:17 PM PDT by ModelBreaker
[ Post Reply | Private Reply | To 100 | View Replies]

To: Tax-chick
I'm reading books on the development of nuclear weapons right now. Evolution doesn't seem relevant to the subject at all.

The question is whether the subject is relevant to evolution. If the scientific principles behind nuclear weapons works, so must the principles behind radiometric dating. Many papers on evolution use radiometric dating as their primary evidence. Now do you see the connection?

105 posted on 08/30/2005 1:06:18 PM PDT by Physicist
[ Post Reply | Private Reply | To 99 | View Replies]

To: Tax-chick

"Consensus science" is always right. If "most scientists" agree, than it must be true. < /sarcasm


106 posted on 08/30/2005 1:18:04 PM PDT by TaxRelief (You have two choices: Convert to Islam or suppress Islam. There is no other option, Mrs. Sheehan.)
[ Post Reply | Private Reply | To 15 | View Replies]

To: knuthom
It has always been important to have reproducible results.

How do you expect to get any government grants with crazy talk like that?

107 posted on 08/30/2005 1:21:36 PM PDT by Straight Vermonter (John 6: 51-58)
[ Post Reply | Private Reply | To 3 | View Replies]

To: Physicist

Today, I am complaining about evrything. Your reply was the most informative on the subject.

Can we talk about global warming instead?


108 posted on 08/30/2005 1:26:02 PM PDT by Cold Heart
[ Post Reply | Private Reply | To 103 | View Replies]

To: Duke Nukum
This is one reason I see science as a quasi-religion. I'm asked to take too much on faith for it to really be science.

You're right. It's a quasi-religion based on faith in the 'scientific method'. And like any other religion it has its share of quacks, hustlers, con men and myth makers. However, science used honestly and with an understanding of its limitations is a great benefit to mankind.


109 posted on 08/30/2005 1:32:41 PM PDT by Donald Rumsfeld Fan ("Memos on Bush Are Fake but Accurate". NYTimes)
[ Post Reply | Private Reply | To 68 | View Replies]

To: ModelBreaker
In olden times at my grad school, there was a professor who had a student run a study and perform a statistical test. The test showed the results were without significance.

The professor kept the data and each following year, she had a new student perform the same test, with the same result. After a few years, her student made an error in calculation and reported the results were significant.

The professor then submitted the study for publication. She included means and standard deviations for the various groups in her study.

Apparently, she was unaware that any reader of the published aricle could verify the statistical test from the means and SDs. One reader dicovered the error and demanded a retraction.

The professor's response? She loudly complained, I should never have included the standard deviations!

110 posted on 08/30/2005 1:36:52 PM PDT by Marylander
[ Post Reply | Private Reply | To 24 | View Replies]

To: LibWhacker
Doubtless, many papers are wrong for the reasons cited and others; but science proceeds nonetheless -- the evidence is all around us.

The paper fails to recognize that results differ in different branches of science. At one extreme is mathematics, which is an entirely cumulative enterprise; once a theorem is proven, it can be used in perpetuity, and many current theorems find their roots in theorems proven more than 2 millennia ago in Greece.

By contrast, in physics, findings are never final. Newton's formulation held up for about two centuries until superseded by Einstein's, which will in due course be superseded as well.

But when you get to the so-called soft sciences (social sciences), the record is more dismal. One of the problems is that there is a trade-off between relevance and reliability. You can have a paper with high reliability but no relevance, and you can have a paper with high relevance but no reliability. What you can never have is a paper with high relevance and high reliability.

Nevertheless, the vigor of our economy is predicated on the power of our technology, which is in turn dependent on the quality of our science. Despite all the weaknesses noted on this thread, no one can deny that we ride around in jet planes, eat and drink safe foods, live in relatively safe dwellings, and trade these messages over a network, using computers and software, all of which are the products of science, from which we can conclude that at least some of the products of science are usable.

111 posted on 08/30/2005 1:40:41 PM PDT by Sarastro
[ Post Reply | Private Reply | To 1 | View Replies]

To: LibWhacker

Assuming, as good scientists do, that there will always be more to learn about most subjects, it is technically accurate to say that all statements about scientific subjects are wrong or at least incomplete. That being said, scientists and engineers who put scientific findings into practice are pretty much right about most things.


112 posted on 08/30/2005 1:45:04 PM PDT by muir_redwoods (Free Sirhan Sirhan, after all, the bastard who killed Mary Jo Kopechne is walking around free)
[ Post Reply | Private Reply | To 1 | View Replies]

To: LibWhacker; DaveLoneRanger
Peer review = Researchers' bias multiplied.

This is not surprising in the least.

113 posted on 08/30/2005 2:02:21 PM PDT by ohioWfan (If my people which are called by my name will humble themselves and pray......)
[ Post Reply | Private Reply | To 1 | View Replies]

To: Physicist

Yes, I do see. However, the fact that the principles behind something are correct does not mean that the experimental structure or the interpretation of data is correct.

I think I'd rather just blow things up.


114 posted on 08/30/2005 2:04:30 PM PDT by Tax-chick (Oklahoma is the cultural center of the universe ... take me back to Tulsa!)
[ Post Reply | Private Reply | To 105 | View Replies]

To: TaxRelief

I thought it was something like that. Sometimes I get confused.


115 posted on 08/30/2005 2:05:23 PM PDT by Tax-chick (Oklahoma is the cultural center of the universe ... take me back to Tulsa!)
[ Post Reply | Private Reply | To 106 | View Replies]

To: LibWhacker

read later bump


116 posted on 08/30/2005 2:09:14 PM PDT by Kevin OMalley (No, not Freeper#95235, Freeper #1165: Charter member, What Was My Login Club.)
[ Post Reply | Private Reply | To 1 | View Replies]

To: Donald Rumsfeld Fan

I like the scientific method. It's an invaluable tool in my every day life but like any tool it can be abused.

Act locally, bludgeon globally, as they are fond of doing.

I just think modern science had institutionalized flim flam. Nothing wrong with it, I guess, if one can get away with it. Just don't expect me to believe anything they say just because they're published in JAMA or whatever.

To thine own self be true...or relatively true.

Maybe the real reason "scientists" despise "psychics" is because the "psychics" cut into the "scientists" margins?

Just a wild speculation, at least until I can fabricate some data and get it published in Science.


117 posted on 08/30/2005 2:10:22 PM PDT by Duke Nukum (To thine own self be true...or relatively true. --Guy Caballero)
[ Post Reply | Private Reply | To 109 | View Replies]

To: Marylander
Apparently, she was unaware that any reader of the published aricle could verify the statistical test from the means and SDs. One reader dicovered the error and demanded a retraction. The professor's response? She loudly complained, I should never have included the standard deviations!

Was this in the social sciences? Her whining sounds like she would support outcome based education and self-esteem.

118 posted on 08/30/2005 3:30:59 PM PDT by ModelBreaker
[ Post Reply | Private Reply | To 110 | View Replies]

To: Cold Heart
Can we talk about global warming instead?

Can do. Here's a thread from yesterday that's still going.

119 posted on 08/30/2005 3:58:43 PM PDT by Physicist
[ Post Reply | Private Reply | To 108 | View Replies]

To: LibWhacker

Well, it depends on what is "right." Most real scientists don't claim to know the truth, only a reasonable approximation of it.

The only people who are RIGHT generally run kooky websites and live in momma's basement. :P


120 posted on 08/30/2005 4:04:13 PM PDT by Constantine XIII
[ Post Reply | Private Reply | To 1 | View Replies]


Navigation: use the links below to view more comments.
first previous 1-20 ... 81-100101-120121-140141-158 next last

Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.

Free Republic
Browse · Search
News/Activism
Topics · Post Article

FreeRepublic, LLC, PO BOX 9771, FRESNO, CA 93794
FreeRepublic.com is powered by software copyright 2000-2008 John Robinson