Science is in a reproducibility crisis: How do we resolve it?

Phys.org ^ | Sep 20, 2013 | Fiona Fidler and Ascelin Gordon

[Olog-hai]

Over the past few years, there has been a growing awareness that many experimentally established “facts” don’t seem to hold up to repeated investigation.

This was highlighted in a 2010 article in the New Yorker entitled The Truth Wears Off and since then, there have been many popular press accounts of different aspects of science’s current reproducibility crisis.

These include an exposé of the increasing number of retractions by scientific journals and damning demonstrations of failures to replicate high profile studies.

Articles in recent days have discussed how the majority of scientists might be more interested in funding and fame than “truth” and are becoming increasingly reluctant to share unpublished details of their work. …

[1malumprohibitum]

I am shocked!

[Moonman62]

The article is about a solution to a problem (see the last two paragraphs), but let’s see how many FReepers use this as an opportunity to bash science.

[ClearCase_guy]

Methodology and statistics courses are one obvious place

I would agree that these fields would help get science into a better place. But, of course, statistics has its own problems. The economy is doing great, right? Unemployment rate is low and often drops, yes? That's what the statistics say.

More than anything, I would like to see an overhaul in how statistics are used -- because they are often used dishonestly. Perhaps, as part of that, science could jump on the bandwagon and undergo some of its own reforms.

[Rocky]

In addition there is currently a push to give researchers a greater incentive to publish their data by making scientific datasets citable contributions to the scholarly record and with associated journals such as GigaScience and Earth System Science Data.

Do you suppose that they can persuade Michael Mann or the University of Virginia to release the data behind the famous global warming hockey stick?

[LibLieSlayer]

Science is now an art of whoring.

[LearsFool]

Science is to be an honest inquiry, a tool to serve us. When the checks against dishonesty (i.e. reproducibility, disprovability, debatability, etc.) are overthrown, not only is this tool rendered incapable of serving us, but it will be seized and used as a weapon against us.

[agere_contra]

If it’s not reproducible, they can bash it all they like.

[Hostage]

Why are you bashing conservatives who criticize scientific misconduct or unethical or politicized constructs couched in pseudo-science? Is criticism by conservatives of Global warming opportunistic? There have been many insightful exchanges of views and criticisms of science on FR over very many years.

I am a scientist. I hold a PhD from one of the top 3 university departments in the world in my subject matter which is one of most difficult hard science subjects. I have been involved in myriad studies, trials, experiments and publications. I’ve written millions upon millions of dollars worth of federal and private foundation grants.

I find the polemic of FR in passionate exchanges of science discourse to be refreshing and revealing. There are many scientists here on FR that come clean with their unobstructed views on misapplication and ill motive of scientific research, especially federally funded research.

I read the ‘solutions’ posed by this ‘crisis’. First, I am not persuaded that there is a crisis above that which has existed for previous decades since the explosion in federally funded research.

Here is the gist of the so-called solution proposals:

“The proposals and initiatives mentioned above draw attention to improving methodological protocols, and require a more thoughtful approach to statistical reporting practises.”

Statistical reporting practices or practises indeed. Aside from the fact that the misapplication of statistics is not at all a new issue or ‘crisis’, the suggestion of a more thoughtful approach to statistical methodology has been an issue in science for centuries. And the solution has nothing to do with shallow suggestions of mediocre bureaucracy, namely reproducilibity index reporting, regulation of funding bodies or ‘random audits’.

And I note this work product of phys.org appears from the ‘soft science’ of psychology which is a field largely of lazy discounted PhDs who are the biggest abusers of statistical methods because they don’t understand and mostly do not possess the capacity or scientific acumen to absorb the mathematical philosophy of statistics. They are no better that political poll con artists. How’s that for a bit of Freeper bashing?

Statistics is like a scalpel, the outcome or result is entirely dependent on the training and experience of the surgeon.

And here is the heart of any so-called ‘crisis’:

“We might broadly consider these to be issues of researcher integrity.”

Yes indeed, this is the crux of all scientific pursuit, political debate or legal justice, the search for truth.

“What is truth?” John 18:38

One cannot legislate an effective or impose an enduring system to regulate ***sound moral character*** in human beings which is vital for matters involving the search for truth. It must come from within.

It’s clear your comment targeting Freepers for ‘bashing science’ has no merit. If you were experienced in Freeper discourse over the decades, you would know that some of the finest legal, political and scientific minds write commentary on FR, and do so for free while donating to keep FR operating. You should respect this fact.

[from occupied ga]

FReepers use this as an opportunity to bash science

Science doesn't have a credibility problem "pseudo-science" (like global warming, cold fusion, creationism, etc.)have credibility problems. Science starts with the observations and shapes the theory to fit the observations. Pseudo science starts with the conclusion and manipulates "facts" and falsifies observations to fit the conclusions.

[Hostage]

> “That’s what the statistics say.”

No, that’s what people say that statistics say. Statistics do not lie, people lie and some people lie using invalid statistics.

It’s the same type of false argument used in the framing the politics of gun control; “Guns kill people, therefore regulate guns”.

The scientific field of statistics has for more than a century pointed out the misuses of statistics by people calling themselves scientists. It’s all there in the literature.

No regulation or system is going to cure or mitigate human misconduct. The scientific marketplace regulates itself by discounting names of so-called scientists who misuse methods and mislead readers and consumers.

If you were deeply aware of the true players in science, you would know them by name and reputation. But there are many posing as scientists who in fact are nothing more than con artists. And there will always be con artists just like the fact there will always be prostitutes. It is up to you, it is up to each of us to see who is truthful and who is not and that can only start by holding truth as a value.

[sima_yi]

Couldn't agree more. The quote from Benjamin Disraeli (among others) should have been

"There are lies, damned lies, and the MISUSE OF statistics."

[Alas Babylon!]

I agree—to an extent. The biggest problem we have in science is the politicization of science by the Left and Leftwing environmentalists these past 50 years.

The fact that they could ever declare “consensus” over something as complex and challenging as anthropogenic global warming (AGM) is extremely important to speak out on.

Scientists who tried to say “Wait just a minute!”, based on need for more facts and proof where singled out as “Deniers”, a very politically charged term that was meant to conjure up images of neo-Nazi supporters and racist skin-heads. A deliberate accusation that had no part in honest scientific debate.

Anyway, I know not all scientists participated in this. In fact very few. But for those that did, and let the media promote them as the true, global opinion of science was and is shameful. Moreover I really see no change at this point. Just look at the Latest Issue of National Geographic Magazine:

The fear-mongering in the Rising Seas issue is palpable. And many scientists are quoted in insisting AGM is real and will cause the sea to rise 125 feet, at least.

[norwaypinesavage]

The Scientific Method has been around for millennia, and it is tried and true. It's quite simple: Propose a theory. Test the theory with experiments. Publish all results and data. Successful confirmation results in the theory becoming dogma.

Recent government funding has resulted in much corruption of science. Politicians don't seek truth, they seek sycophants.

[norwaypinesavage]

The Scientific Method has been around for millennia, and it is tried and true. It's quite simple: Propose a theory. Test the theory with experiments. Publish all results and data. Successful confirmation results in the theory becoming dogma.

Recent government funding has resulted in much corruption of science. Politicians don't seek truth, they seek sycophants.

[BenLurkin]

many experimentally established “facts” don’t seem to hold up to repeated investigation.

Huh.

[BillM]

“The scientific field of statistics”
Statistics is NOT science. It is mathematics.
Science deals in certainty.
Statistics deals with probability.

[Marylander]

I stopped believing “studies” 10 years ago .. for the reasons listed in the article.

[zeugma]

<zeugma claps>

[Hostage]

Statistics is mostly definitely a science as its methods conform with scientific method. Each method is tested against another standard.

Science deals with probability and statistics in that date collected is often imprecise and to estimate paramters requires statistical methods. Scient