How Much Scientific Research Is Actually Fraudulent?

Reason ^ | 7-9-21 | Ronald Bailey

[DeweyCA]

Fraud may be rampant in biomedical research. My 2016 article "Broken Science" pointed to a variety of factors as explanations for why the results of a huge proportion of scientific studies were apparently generating false-positive results that could not be replicated by other researchers. A false positive in scientific research occurs when there is statistically significant evidence for something that isn't real (e.g., a drug cures an illness when it actually does not). The factors considered included issues like publication bias, and statistical chicanery associated with p-hacking, HARKing, and underpowered studies. My article did not address the possibility that the lack of reproducibility could be because a significant proportion of preclinical and clinical biomedical studies were actually fraudulent.

My subsequent article, "Most Scientific Findings Are False or Useless," which reported the conclusions of Arizona State University's School for the Future of Innovation in Society researcher Daniel Sarewitz's distressing essay, "Saving Science," also did not consider the possibility of extensive scientific dishonesty as an explanation for the massive proliferation of false positives. In his famous 2005 article, "Why Most Published Research Findings Are False," Stanford University biostatistician John Ioannidis cited conflicts of interest as one factor driving the generation of false positives but also did not suggest that actual research fraud was a big problem.

How bad is the false-positive problem in scientific research? As I earlier reported, a 2015 editorial in The Lancet observed that "much of the scientific literature, perhaps half, may simply be untrue." A 2015 British Academy of Medical Sciences report suggested that the false discovery rate in some areas of biomedicine could be as high as 69 percent. In an email exchange with me, Ioannidis estimated that the nonreplication rates in biomedical observational and preclinical studies could be as high as 90 percent.

The possibility that fraud may well be responsible for a significant proportion of the false positives reported in the scientific literature is suggested by a couple of new Dutch studies. Both studies are preprints that report the results of surveys of thousands of scientists in the Netherlands aiming to probe the prevalence of questionable research practices and scientific misconduct.

Summarizing their results, an article in Science notes, "More than half of Dutch scientists regularly engage in questionable research practices, such as hiding flaws in their research design or selectively citing literature. And one in 12 [8 percent] admitted to committing a more serious form of research misconduct within the past 3 years: the fabrication or falsification of research results." Daniele Fanelli, a research ethicist at the London School of Economics, tells Science that 51 percent of researchers admitting to questionable research practices "could still be an underestimate."

In June, a meta-analysis of prior studies on questionable research practices and misconduct published in the journal Science and Engineering Ethics reported that more than 15 percent of researchers had witnessed others who had committed at least one instance of research misconduct (falsification, fabrication, plagiarism), while nearly 40 percent were aware of others who had engaged in at least one questionable research practice.

In a blistering editorial earlier this week, former editor of the medical journal The BMJ Richard Smith asks if it's "time to assume that health research is fraudulent until proven otherwise." Smith calls attention to a systematic review of randomized controlled trials recently submitted to the journal Anaesthesia by British anesthetist John Carlisle. He found that of the 153 studies for which individual patient data were available, 44 percent had untrustworthy data and 26 percent were what he called "zombie" trials whose results are animated by false data. Carlisle pointed out that many of the zombie trials came from researchers in Egypt, China, India, Iran, Japan, South Korea, and Turkey.

In an editorial, Ioannidis observes that the zombie anesthesia trials added up to "100% (7/7) in Egypt; 75% (3/ 4) in Iran; 54% (7/13) in India; 46% (22/48) in China; 40% (2/5) in Turkey; 25% (5/20) in South Korea; and 18% (2/11) in Japan." Taking the number of clinical trials from these countries listed with the World Health Organization's registry and extrapolating from the false trial rates identified by Carlisle, Ioannidis estimates that there are "almost 90,000 registered false trials from these countries, including some 50,000 zombies." Consequently, he concludes that "hundreds of thousands of zombie randomised trials circulate among us." Since randomized controlled trials are the gold standard for clinical research, Ioannidis adds, "One dreads to think of other study designs, for example, observational research, that are even less likely to be regulated and more likely to be sloppy than randomised trials."

In his BMJ editorial, Smith cites the work of Barbara K. Redman, author of Research Misconduct Policy in Biomedicine: Beyond the Bad-Apple Approach. During a webinar on research fraud, Smith reported that she insisted "that it is not a problem of bad apples but bad barrels if not of rotten forests or orchards." Redman argues, according to Smith, "that research misconduct is a systems problem—the system provides incentives to publish fraudulent research and does not have adequate regulatory processes." The research publication system is built on trust and peer review is not designed to detect fraud. Journals, publishers, funders, and research institutions have little incentive to check for fraud and a big disincentive against damaging their reputations by retracting studies.

So what can be done to stem the tide of apparently fraudulent research? Ioannidis suggests that one useful step would be to require that all datasets must be made available for reanalysis by other researchers. That is how Carlisle was able to identify untrustworthy and zombie anesthesia studies. Some hard thinking needs to be done about how to change incentives from publishing studies to discovering the true things about the world. For the time being, Smith may be right that "it may be time to move from assuming that research has been honestly conducted and reported to assuming it to be untrustworthy until there is some evidence to the contrary."

Nevertheless, I still agree with Ioannidis, who once told me, "Science is, was, and will continue to be the best thing that has happened to human beings."

[DeweyCA]

When people say, "Science says," remember that "science" says nothing. Scientists (many of whom are biased, greedy and deceitful) are the people who are telling us their "results." Think of all of the hoaxes in global warming and other leftist scare-mongering.

[UnwashedPeasant]

57% of statistics are fake, including this one.

[nickcarraway]

We have a reproducibility crisis in science.

It seems like the U.S. is worse than other countries, lie say Israel.

[DannyTN]

How do you know if the research into research fraud is fraudulent?

[tired&retired]

Let China steal the fake science..

[ALPAPilot]

I wonder if there is a correlation between bogus science and the leftward turn of academia. Liberals ruin everything.

[Fresh Wind]

Scientists say their quest is for knowledge. In reality, their quest is for research grants.

[jimfree]

Governments and foundations offer bounty for certain kinds of research. Schools and industry offer positions and titles. Committees offer awards. Scientists seek these things.

[AZJeep]

Scientists are as greedy as other people.
To make most money, you need to generate some scare - global warming made a lot of millionaires!
There is also a need for publications-to show you are doing anything, you need to generate publications, whatever the quality of them is.
Lot of science is totally useless. When I questioned some scientist on the value of some science (actually legit science), I got interesting answer(s):
“There are two possible answers to this question:
One is that somebody sometime in future could find an use for this. (Unlikely in this case!)
The other is that this has its internal beauty, like painting or music, and few people can delight in this.” Very few people, I would say.

[FreeperCell]

All of it is bad. All of it.

[USCG SimTech]

Cherry-picking information worked so well in the realms of politics and society why not industry, sciences, medical, everything? People are lazy and won’t research, double check. There are zero consequences for all this fraud so it will continue to amplify at a blinding pace.

[momincombatboots]

The same people who are rightly critical of climate science are in love with covid science. Show me the evidence, they say. If they only knew how evidence is manipulated. Nonetheless, no amount of truth can convince them.. their lord and savior is the government pushed vax... and the next one, and the next one. Needle kneelers... i am okay with that, but the needle pushers are just leftists.. even here on fr.

[originalbuckeye]

A lifetime ago I was working in arguably the best hospital in the country. One of the doctors in my department was making ‘great strides’ in research in the area of one type of cancer. Shortly thereafter it was found that he had faked his cell lines. He lost his job, but was still able to work (in another area of the country). He was lifetime banned from research.

I will never automatically believe research from anywhere. Especially if it goes against common sense.

[Dutch Boy]

Here is my test if science is real.

1. Is the result useful politically?
2. Is it being used politically?
3. Who does the result profit?
4. What are the goals of those that profit?

[one guy in new jersey]

Agreed.

[LambSlave]

When I did biomed research decades ago, I pointed this out many times to our senior department chairs. This does not have to involve fraudulent intent. The problem is publication bias, and we definitely saw this in several classes of drugs. We also saw bias in surgical interventions, but that is not due to publication bias.

If you do a study showing a drug has an effect, you can get that published; if you do a study showing it has no effect, you cannot. The publications only have bandwidth for so many articles per month, so they will cherry pick the "most significant" results submitted, which will not only be the ones that show something "works" but the bigger or more surprising the results the more likely to publish. If ten people at different locations do the same study, and one sees an effect, they will publish that study, the others will move on to other projects until they complete a study that is "publication" worthy, but no one will be aware of the lack of effect or be able to use this information for statistical purposes.

[Clutch Martin]

Monies for fish fart studies is non-refundable...

[metmom]

Ping

[Robert A Cook PE]

It is sobering that the “results matter” so much.
But, it is the “peer review” half of the coin that is never discussed.

Peer review takes time, LOTS of time spent “on the other guy’s research” and NOT “your own research”, right?

But, there are (now) NO REWARDS, NO RECOGNITION for “good, quality peer review”, and MUCH REWARDS for “quick” and complacent peer review, for acceptance reviews of “wanted papers” and “desired conclusions”. And MANY, MANY rewards for two other things equally troubling:

(1) Harassing and delaying or denying papers from “enemies” of your own research, or research that disturbs your field and hurts others in your field.

(2) Prompting and promoting and fast-tracking and accepting and publishing results (in your field) that DO promote your friends’ ideas, fast-tracking and casually reviewing ideas and papers that DO promote your field and PROMOTE your own (future) funding and your friend’s funding and their promotions.

Method +
Motive +
Money +
Morals +
Mistakes.
= Opportunity for Errors, Opportunity for and Seduction of the Mind.

The only person who an prevent this from happening is the 6-8-12-16 editors of the “scientific journals” who assign the people for peer-reviewing, and who accept the results of that peer-review. Good reviews, AND bad reviews. Or who selectively assign peer-review requests to squash ideas and decisions and results they (the editors) DO NOT WANT APPROVED.

SO, the solution?

Keep the peer review selection anonymous, as we do now. BUT! After acceptance and at publication, publish and praise those who peer-reviewed the paper as equally as the AUTHORS. Include them in the document, include the data in the document (perhaps not “printed” but archived at a minimum accessibly). If a reviewer does disagree with the paper - and that will happen too! - then INCLUDE the disagreement statement or action. (Probably not a full rebuttal, but as a “dissenting reviewer” at a minimum. That is, after all, how the Supreme Court cases publish their agreements, AND their dissents!)

Honor and Recognize every paper that a man or woman peer reviews as a “paper” in its own right, and include peer-review activities as valuable for promotions and pay and tenure (the real driving forces for “bureaucratic scientists and administrators anyway!)

We have seen now the way the entire peer-review process has been corrupted in the climate catastrophic anthropogenic global warming schemes: papers, industries, funding, politics, promotions, selections and labs and firings and publications. (CAGW) Editors and publication selections deliberately corrupted and peer-review (pal-review) are epidemic. And now: Biomedical, psychological (for a long time, and pharmaceutical.

[NCLaw441]

Everyone has biases as well as hope for recognition and fame. This leads to “scientific” predictions that most often do not come true. Climate science is a great example:
https://cei.org/blog/wrong-again-50-years-of-failed-eco-pocalyptic-predictions/