Slowly Strangling America's Golden Goose

From CATO blog:

Chuine et al. claimed to have developed a method for estimating summer temperatures in the French wine region of Burgundy from 1370 to the present based on the dates that grapes were harvested. Using this method, the authors asserted that the summer of 2003 was the warmest summer in Burgundy since 1370. The study was offered up as yet one more piece of evidence that global warming is running amuck.

LINK

Reply to “Ten Simple Rules for Getting Published”

Eric Grosch

Rule 10 for getting published [1] carries advice to publish in journals of high impact (high citation rate). Riding the coat-tails of eminent, high-impact journals is good marketing, but the task is easier said than done, because the higher the impact is the greater is the competition for print space and the more likely the editor is to offer unhelpful feedback, such as a statement on a form letter that he rejects many worthwhile manuscripts for lack of space. Good science may appear in the pages of journals of many degrees of impact. In support of that notion, current impact factors [2] appear in Table 1 for each of the journals (or successor—Am J Epidemiol continued J Chron Dis) cited in this essay (see References).

Table 1

Impact Factors for Journals Referenced in This Essay

Yet, a journal's high eminence and high impact may bespeak its rigid orthodoxy, rather than its high quality. Rule 10 may hold for journals, such as PLoS Computational Biology, in which objective science, evidence, and the GIGO (“garbage in, garbage out”) principle count for something. Eminence-based medicine [3,4] too often substitutes—and poorly—for evidence-based medicine [5]. Altman deplored poor medical research [6], which too often appears in high-impact medical journals, and suggested, “incorrect procedures . . . can be hard to stop . . . from spreading . . . like a genetic mutation” [7]. Consensus in medicine [8] too often permits false doctrine to masquerade as “standard of care,” just as an ad blitz may build a public consensus on specious claims that favor sale of a certain brand of snow tire [9]. Medical science and its “opinion-leaders” were arguably tardy in complying with Rule 6, good science [1], in recognizing Helicobacter pylori in peptic ulcer disease [10,11], thrombolytic therapy for myocardial infarction [4,12], questioning post-menopausal estrogen [10,13], and preventing thousands of crib deaths by rejecting Benjamin Spock's high-impact advice to lay babies prone [14], among other instances [15].

In medical journals, eminence-based medicine [3,4] predominates, and censorship by editors, in attempts to save face, may impair the vitality [16] and self-correction [17,18] of science and the protection of “the literature and the reader from identifiable error” [19], despite editorial lip service to “evidence-based medicine” [5].

Helpful first steps to remedy the current malaise might consist of prompting editors of scientific journals, of all levels of impact, to improve peer review by encouraging substantive dialogue [20], by adhering to logic [21,22] and to valid statistical inference [23–25], by encouraging authors to provide readers access to raw data [7,26–31], the better that readers might verify or challenge published conclusions, by issuing to editorial peer reviewers a “plea for rigor” [32] and diligence [33] by requesting them to “state the rationale, and present the evidence, for exceptions taken to the manuscript” [32], and by incorporating the dialectical scientific brief [34], rather than by perpetuating current inequities: a) for each hour put in by a journal reviewer or editor, the author puts in about seven hours... [35]; b) the average time spent reviewing a paper is less than two hours in medicine [36]; c) the editor invariably defends the reviewer's call. After all, who are we to question the decision of someone who may have devoted much time to the manuscript [37]?

High-impact medical journals too often nurture sacred cows by taking in and putting out orthodox garbage and rejecting innovative pearls. Then the Institute of Medicine wonders why 44,000 to 98,000 patients per annum die of preventable medical errors in the hospitals of the United States [38].

LINK

Peer review: the Holy Office of modern science

Keywords: electronic communication, objectivity, peer review, refereed journals, research grants, science funding, science publishing, subjectivity.

MACIEJ HENNEBERG ^{Note 1}Department of Anatomy and Histology, University of Adelaide, Adelaide 5005, Australia, mhennebe@medicine.adelaide.edu.au

Received February 17, 1997, published February 20, 1997

Summary: A brief historical overview of the origins of peer review reveals that it is neither the best means of evaluating contributions to science nor the one most commonly used during the period in which the modern scientific method developed. Throughout history, most scientists published their views without formal review and peers published their criticisms openly. It is argued here that peer review as now undertaken by most scientific journals stifles scientific communication, slows the advancement of knowledge and encourages dishonest behavior among referees. Alternatives to peer review that have already been used by some journals and funding bodies are described. Since these alternatives have proved themselves in practice, the now commonly practised form of peer review can be abandoned or modified. Electronic communication can facilitate this process.

Science, communication, society
Science is a specific collective human activity (Hull 1988). It is about acquiring an understanding of the world that is of practical value. It is different from both commerce and ideology and hence its practice in society has to be controlled by a separate set of rules and behavioral norms. Exchange of information and opinions among practitioners of science is crucial to its progress. Results of scientific research are useless unless they are communicated to other scientists and the public at large.

Presentation by scientists of the results of their research to others, be the work academic or industrial, is thus essential to the scientific enterprise. Because the nature of scientific work, at least according to Karl Popper (1972), is to approach the truth through a series of increasingly accurate or useful approximations, no piece of research can be held to be definitive. All results are transitional and approximative. Although scientists strive for perfection, by the very nature of their endeavor they cannot achieve perfection. It follows that neither a paper presenting new results of observation or theory nor a proposal for a new investigation can be perfect. They can only be “good enough.” One of the paradoxes of science, which purports to measure various phenomena with great precision, is that it cannot define a precise measurement of “good enough.” Assessment of what “enough” means is left to the judgement of individual scientists. Hence nonspecialists who cannot reach an informed opinion about some theory or discovery often tend to be guided by the consensus opinion of specialists.

Leaving assessment of research results and theories to the judgement of individuals, however well informed in a particular field of expertise, introduces a factor of subjective opinion into seemingly objective science. The ancient Greeks, who originated science as a specific endeavor based on logical argument and empirical tests, realized its susceptibility to subjective opinions. The Greek way of dealing with possibly biased opinions was to conduct open debates between scientists, to produce mathematically exact descriptions and to run empirical tests of statements made by learned persons. Broad discussion and empirical testing were the foundations of learning. Sometimes these were taken too far, as for example in Hellenistic Alexandria, where curious scientists conducted vivisections on condemned criminals (Persaud 1984). Roman ethics put an end to such practices and even forbade dissections of deceased humans. In Roman times, learning became more dependent on written texts than on observation of nature and theoretical arguments. Sanctity of the written word was entrenched by medieval European scholastics.

Science as a human endeavor is also open to social pressures expressed as moral and ethical norms. All human activities need to be organized and regulated by norms of behavior that constrain actions of individuals and attach value to decisions. In other words, individual opinions and actions must be censored and decisions justified.

There exists a basic contradiction between the free search for ideas and their empirical testing on the one hand, and a priori norms of behavior on the other. In the ideal world, free flow of ideas and unrestricted empirical testing should result in the greatest accumulation of knowledge. Empirical tests require time and resources that must somehow be provided by society. No economy is unlimited in size and, hence, no economy can support all possible experiments scientists might think of. Limited resources require that somebody decides which experiments are to be done and which not. Such decisions necessarily involve an element of subjectivity and are constrained by social norms. Scientists in their quest for a practical understanding of the world must navigate narrow straits between social conventions, economic reality and their own human limitations. Not least among these latter are egotism, greed and plain fear.

Historically, it seems that gentlemen-scientists with an elevated social position and independent means, fared the best in science. In the 16th century, at the dawn of the modern era, one such individual, the son of a middle-class family, holder of the respectable office of Canon of the Frombork Cathedral, published a book on the revolution of celestial bodies that revolutionized understanding of our place in the universe. The man was Nicholas Copernicus. His accomplishment was simple: he moved the Earth from the center of the world onto an orbit around the Sun. Being supported by the endowments attached to his post and having received a thorough education at Polish and Italian universities, he had the time and knowledge to conduct astronomical observations and to write a lengthy manuscript in relative peace. He realized the revolutionary nature of his work and dreaded the opinion of many learned colleagues as he clearly spoke against the then accepted views. And yet the book was published. It would not be possible today to publish a book full of minor errors and simplifications and arguing something so ludicrous as a complete reversal of a consensus of well-established authorities. Would it also be possible for a modern scientist to obtain a grant for a proposal based on the supposition that the major publications on the subject are wrong and that he will prove it by conducting observations from the roof of his residence? Absolutely not. Peer review protects us against such lunacies.

Debate of scientific theories and reinterpretation of results is a core of science. This debate benefits from the widest possible participation. This was well recognized by founders of scientific journals in the early centuries of the modern era. Scientific journals and published transactions of learned societies provided means of relatively fast and broad communication of ideas among scientists. Published works were there for everyone to read and it was open to everyone to publish counter arguments. Transactions of many societies included the text of both presented papers and the ensuing debate. All points of view were there in print for everyone to see and comment on. Reading such printed accounts could be bewildering for inexperienced readers, but sharpened their powers of reasoning as each argument had to be absorbed, evaluated and compared with other arguments.

With time it became obvious that not all written matter submitted to journals for publication or to societies for presentation could be published. A certain standard of quality had to be applied. Editors of journals and officers of societies made judgements as to what is acceptable and what not acceptable. Junior or less well-known scientists were not allowed to present their papers at the meetings of various societies, but their papers were introduced by recognized fellows who themselves were elected to fellowships by older and well-established colleagues. Fellowships of the Royal Society and many national academies are awarded in this manner even today.

Editors of journals and fellows of learned societies in the 19th century were usually broadly educated academics. They felt themselves to be well equipped to judge the quality of practically all research reports and theoretical papers submitted to them for publication or presentation. In making such judgements, they also kept in mind the reputation of the journals and societies, and so they also acted as guardians of publicly accepted ethical norms.

It is only natural for humans to evaluate possible consequences of a public statement before making it. Such evaluation, besides obvious reliance on an individual’s knowledge and experience, often tends to be based on the advice of friends, family members and colleagues. Hence many manuscripts of scientific works were read and commented on by the author’s friends and colleagues before being submitted for publication. Such was the regular practice of Charles Darwin, who tested his ideas on his friends before committing them to print (Desmond and Moore 1991).

Peer review today
As the 20th century dawned, the breadth of scientific endeavor became such that there were hardly any persons who could feel competent to evaluate contributions in more than a limited range of subjects. Thus, editors started asking specialists to evaluate papers submitted to journals. The specialists acted in an advisory capacity, and editors still took full responsibility for their decisions. In order to encourage frankness by referees, editors undertook to ensure that they would remain anonymous to the author. With time, overworked editors transferred all work of evaluating manuscripts and suggesting corrections to referees. Referees took make-or-break decisions, but still remained anonymous. There is a commonly held view that refereeing of papers ensures that they become definitive publications, and that what is published must be true and need not be questioned. This is obviously wrong as referees are as prone to errors as are authors. Although it is less likely that a paper will contain major mistakes after it has been scrutinized by several reviewers, errors in published papers are nevertheless common. The proof lies in a number of corrections published in Errata sections of major journals.

The most common form of peer review today is for an editor to send a manuscript to several specialists asking them to answer standard questions about the paper and to provide written comments. The questions nearly always include the one regarding recommendation to publish “as is,” “with revisions” or “reject.” The process used to referee grant proposals is virtually the same, although instead of “accept” or “reject” referees are asked to assign a score or a grade to the proposal.

Being asked to referee a paper or a grant proposal anonymously is the dream of anybody seeking power—power without responsibility. Referees acting under such circumstances may be tempted to reject, or delay, publication of papers they disagree with, as their pronouncements regarding the quality of work under review are unlikely to be questioned. The same goes for the refereeing of grant proposals (Goodstein 1996).

Since all the thinking about delaying publication of “unwelcome” results is done in the heads of anonymous referees, it cannot be examined directly. It can, though, be teased out statistically. Increased reliance of journals on decisions made by reviewers increases time from submission of a paper to its final acceptance, because referees recommend alterations to the content and form of the manuscript. The number and difficulty of proposed alterations may be proportional to the intention to delay publication. Many journals have adopted the practise of printing the “date submitted” and the “date accepted” as a footnote to a paper. I have examined the delay between submission and acceptance in two journals: the American Journal of Physical Anthropology (AJPA) and the Medical Journal of Australia (MJA). In 1980, when AJPA first published the information, the average waiting time from submission to acceptance was 6.9 months (Median = 4.8, SD = 4.5) whereas in 1996 it increased significantly (P < 0.01) to 15.7 months (Me = 11.3, SD = 9.8). Papers published in MJA in 1984 had an average waiting time of 3.5 months (Me = 2.0, SD = 3.3), which increased significantly (P < 0.05) to 5.6 months (Me = 4.4, SD = 2.5) in 1995. It is strange that these increases occurred during a period in which the quality of research work should have improved and the efficiency of mails and other communication systems increased. Slowdown in communication of scientific results is obviously to the detriment of scientific progress and appears linked to reliance on the process of peer review.

The peer review of grants has its origin in industrial practice. Many people propose to do many things but resources are limited. How can a small number of people who hold the purse strings know everything? The answer is that they cannot, and so they must ask specialists. The best solution is to ask people working on problems similar to the one a grant application proposes to solve. They will be able to highlight merits and point out errors of the proposal. Of course they will be more frank if ensured anonymity. Since people working in similar areas usually know each other and either agree and collaborate or dislike each other and compete among themselves, the result of a review of a particular application will depend on considerations external to the application. These considerations, besides likes and dislikes, also include political and financial elements. What being anonymous and having power means to a referee has been discussed above. A good grantsman will write an application in such a way as to avoid being criticized. It will either be bland, proposing an innocuous piece of research, or it will propose to continue what the author has been successfully (in terms of grants) doing before. Any testing of somebody else’s theory or encroachment into someone else’s area of research creates a high risk of failure.

Paradoxically, some good scientists who do not feel inclined to use anonymous powers to manipulate the system, are not keen to act as referees. This happens to the extent that one of the major science funding agencies now coerces people into refereeing grants by saying that it will automatically refuse grants to academics who do not agree to referee proposals of others. For an honest but busy academic there is very little attraction in refereeing a paper or a grant proposal. It takes time and effort to write a well thought-out assessment and, since it is anonymous, no credit accrues to the reviewer even if, in the process, he comes up with a bright new idea. Authors of papers who receive substantial help from referees certainly recognize their indebtedness, as they often thank referees for their input in the “Acknowledgments” section of their papers. Referees of grant proposals find themselves in an even worse predicament. Since they referee grants in their own area of interest, they may be strengthening their own competitors if they provide honest criticism that can be used to improve the proposal.

In general, peer review stifles scientific enquiry both by subjecting authors to anonymous critics whose comments cannot be directly challenged, and by self-censorship by authors who hesitate to state the exact rationale or goals of their proposed research for fear that it would not pass the scrutiny of their colleagues. The word colleagues describes referees more truthfully than the word peers, because, although all scientists have a right to do research and express opinions in any area, scientists are not all equal in experience, talent and diligence. Those who are experienced, talented and meticulous, rarely have peers in the sense of equals. How many physicists of the 20th century would consider themselves equals of Albert Einstein? Nevertheless, an enthusiastic and talented, but inexperienced postgraduate student is going to have her work judged by older, much more experienced professors as if her work had been done by a professor. A bit of understanding and encouragement, rather than strict criticism would perhaps help to develop a valuable scientist. As it is, she may be discouraged by rejection of her work following peer review.

Alternatives to current peer review practices
These days, everybody complains about deficiencies of peer review, but few believe that anything can be done about it. How such a view could pass peer review is beyond me. Over the years and in various countries systems of evaluation according to principles other than “peer review” have been applied with good results. These systems still rely on opinions of academics other than the author, but these opinions are not anonymous or dependent on reviewers ticking boxes or assigning a score.

Until about 20 years ago, many journals relied on editors to make decisions without formal refereeing of papers. Authors were expected to have their manuscripts read and discussed by their colleagues before submission, and it was customary for well-established academics to recommend to the Editor publication of papers written by younger colleagues or students, although this did not guarantee publication. At the same time, students were free to submit papers without professor’s recommendation. Editors, as they became conscious of good work by particular academics, approached them with invitations to publish in their journals. This tradition continues in a limited way even today. For example, only two years ago a respected French colleague approached one of my students to publish his freshly completed PhD thesis in a series of which he was an editor. Such an invitation does not mean that the manuscript will not need to be edited based on direction of the Editor and comments of whomever he has had review the manuscript. The decision, however, is very much out of the hands of peers. Although an editor is, like an anonymous peer reviewer, merely human, his name is known to authors and he has some recognized general policy for his journal. Furthermore, he has a clear interest in the reputation of the journal he edits. These goals are not in conflict with the specific research goals pursued by authors who may wish to publish in his journal.

The editorial system used by leading journals such as Nature run on somewhat similar lines. In the first instance, papers submitted to Nature are evaluated by its editors and only a small number that survive editorial scrutiny are sent for peer review, which therefore takes the form of technical advice to the editor. The extent of editorial control over the process of publishing papers in Nature is evident from the fact that the time from submission to acceptance has not changed over the last 12 years. My calculations based on 283 research papers indicate that the mean delay between submission and acceptance was the same in 1984 (3.1 months, Me = 2.9, SD = 1.7) as it was in 1996 (3.1 months, Me = 2.8, SD = 1.8). The system used by naturalSCIENCE is constructed along similar lines.

The need to allocate research monies unavoidably limits the freedom of scientific exploration, with the result that not all scientists get to pursue the research they want. However, the less the process of allocation of funds stifles scientific freedom the better. Giving money for three-year bits of circumscribed research, a practice that seems now to be quite common, is the worst means of fund allocation. It seems to have arisen from the need to give peace of mind to administrators who cannot make too big a mistake by giving away funds in dribs and drabs to projects having tight schedules and detailed budgets. How can one budget for a discovery? Until the end of 1995, an alternative system was used by the major South African government agency the Foundation for Research Development. In a modified form, combined with project-oriented funding, the same method is still used under the new democratic South African government. The system is based on the principle that good scientists produce even better science when given resources. It relies on a scientist’s track record. A funding submission consists of a Curriculum Vitae with a description of research achievements, concentrating on the most recent past. The candidate is invited to name internationally renowned referees, although the agency may use other persons as well. The submission is evaluated by several international referees in order to establish whether the applicant is an international leader in his field (A), a leader in a specialized area (B), or a scientist who makes regular internationally recognized contributions to his field (C). If none of those descriptions applies, the person is classified D (inactive, no funding will be allocated). There is a special provision for young scientists who have only just completed their doctorate but have proven their worth in other ways (Y). Formerly, an academic was reevaluated every 4 years. Funding was allocated for 4 years based on a category and a one-paragraph general description of the planned research. Now, a more detailed research plan is usually required. The idea is simple—trust a researcher and she will produce results. Similar methods of funding include giving a renowned researcher an endowed chair with research funds attached, or establishing a university research unit for an eminent scholar, which will receive university-funded equipment and staff. In case of younger people, funds such as scholarships or postdoctoral fellowships are allocated by a process based on recommendations by supervisors and evaluation of previous work and experience by variously constituted panels.

It seems ludicrous today to mention past systems in which academic appointments carried a salary designed to cover research expenses, or departmental budgets including lines for research equipment, consumables and travel. And yet, under such arrangements, good research was done because academics were genuinely interested in what they were doing and did not have to rack their brains for ideas that looked acceptable on a grant proposal. Most of the Nobel Prizes won earlier this century were for work done under such circumstances.

There are ways, short of abolishing it, to make peer review less obstructive. The simplest is to take away from reviewers the power to make decisions on a paper or a grant. Instead, they should be asked to provide substantive written comments but not to tick a box recommending a concrete decision or to assign a score. Then the editor or a panel of a grant agency will have to make a decision based on their own assessment of the work and specific points raised by the referees. More work for decision makers, but a fairer deal for authors. The other way is to make referees' names known to the author. This introduces subjective bias, but of a different kind—if a referee wishes to criticize the work he can still do it and it will carry more weight as his personal reputation is at stake. Every time I referee a paper from which names of authors have not been removed (for the purpose of “blind” refereeing), I insist that the editor reveals my name to the author, even if my comments are very critical. This ensures equality—I know her name, she knows mine.

Generally, it seems that the authors should be free to publish their results and conclusions provided that they are presented in a technically correct manner (which editors can check for themselves), and colleagues should be free to publish their criticisms and derive any credit that may be due for innovative comments, and for exposing themselves to debate. Introduction of electronic means of communication goes a long way toward making possible this ideal of free exchange of scientific information. Cost of electronic publishing is lower and hence the volume of exchanged information can increase. Traceability of electronically published pieces is good, and hence appropriate credit can be given to authors and their work can be formally quoted.

LINK

Peer Review is Censorship and Intimidation

Peer review is a form of censorship, which is tyranny over the mind. Censorship does not purify; it corrupts.

Peer review is often assumed to be a purification process. There is no such thing as purifying science. Scientific knowledge must continually evolve. Like all truth, no one can arbitrate it; it must speak for itself through the evidence. There is a lot of junk science and trash that goes through the peer review process.

Prior criticism is always of value. But the author has to decide what to do with it. Imposing thought onto someone is a guaranteed corruption.

The purpose of review should not be to determine fact. Three scientists cannot determine fact for two million others.

What should replace peer review is letting editors decide what to publish based on guidelines which are openly and accountably created. Editors don't need to look at technical details. Let all of the scientists evaluate the technicalities instead of three.

There is a natural inclination to review and criticize any document before it is published. Everything from fiction to news is handled that way. And it is censorship which restricts all forms of publishing to fit in some type of box. In industry, the owner has a right to do that. But there is no owner for science. Its product is supposed to be an evolving truth which does not fit into a subjective box. Therefore, the reviewing should be limited to suggestions, not a shaping and controlling process. After the suggestions, let scientists include their errors with their worth, particularly since there is seldom agreement as to what is error and what is worth in science.

Scientists are so intimidated that they cannot oppose official corruptions in science. One of the main causes is peer review.

If peer review were open and accountable, there might be a small chance of correcting some of the corruptions through truth and criticism; but the process is cloaked in the darkness of anonymity.

There is no place for secrecy in science after the research is done. A laboratory needs some protection from interference while it is working through the challenges, but the evaluation process cannot produce truth through secrecy and unaccountability.

You might assume that there are no official errors in science. Due to the exploitive and corrupt process, nearly everything in science has official errors within it. There is nothing in science entirely free from errors, while a culture of protecting and exploiting the errors creates an official reality which cannot be opposed.

Relativity was the most obvious and extreme example. It was imposed upon physicists, and none were allowed to dissent, as explained on other pages.

A recent parallel in biology is prion proteins as the supposed cause of Scrapies-like diseases. But unlike physics, the evidence in biology is far more available and less abstract. So prions are a much more open defiance of principles and standards. Prion promoters say or imply that the laws of natural selection can be contradicted on the basis of their junk science research.

Debunkers of corruption say no one conspires such things. So an explanation of how it works is necessary.

The first fact involved is that science research is extremely demanding, because the unknown does not easily yield information to technological gimmickry. The second most significant fact is that there is an extreme deficiency of abstract understanding among the world's elites including scientists. They do stupid things and make stupid statements which conflict with everything they are suppose to know on the subject.

They react by demanding that no one question their errors. That demand becomes a rule for power elites in all areas. Scratch each others backs and never oppose the group, or the result is to get shoved out.

In science the result is that errors keep getting compounded, while a pretense of normalcy is maintained. Scientists talk around obvious corruptions, as if they could not see an elephant in a bathtub.

Ultimately, there has to be external accountability for corruptions. In science, the public needs to be creating accountability through criticism.

There is a general assumption that peer review improves publications. Supposedly, deficiencies are corrected, and wording is clarified. It's a pipe dream. Purifying is how complex results are destroyed. It's like redesigning an elm tree or improving the Edsil. It isn't an elm tree or Edsil afterwards.

The mentality seems to be developed when students are writing term papers. They quote publications as being fact. So science publications are supposedly quotable fact. They never are. The pretense of fact destroys the process of evolving knowledge.

Having two or three experts modify someone else's work assumes that research should be perfected before being presented to everyone else. So the rest of the scientists have two or three persons doing their evaluating for them. Scientists are all supposed to be capable of doing their own evaluating. Scientists need to see the deficiencies as well as the value in research.

Because of these forces, there has been a devolution of science publications from a complete description with specifics to something resembling a news article or propaganda sheet. The specifics and details are gone, and all one sees is rehashed opinions.

There is no constructive form that peer review could take. Science publications should use their professional staff, which they already have, to evaluate basic standards only. All of the rest of the limitations need to be visible to everyone.

Addendum

In the arguments over global warming, some persons, such as Monbiot, take the position that if it is not peer reviewed science, it is not relevant to the subject. Since the IPCC represents the peer reviewed science, no one can question the IPCC conclusion that humans are causing global warming.

Monbiot's claim that if it is not peer reviewed, it is not science, is an arbitrary and useless way to define science, because science hasn't address many of the questions that people need to address in life. Science cannot prove that water is wet (or anything else), yet people need to know that it is. And science has become at least as corrupt as any power structure in society, which means everything about it needs to be criticized.

To eliminate external criticism of science by pretending that peer review purifies science is about the functional equivalent of preventing anyone from questioning Hitler's power. Nothing is supposed to be above external criticism; and the tendency of the public to not criticize science is one of the main reasons why it has become so corrupt.

Then, no one with half a brain would define science in terms of peer review. Science is a method of proceeding and a standard which shows evidence for basic questions. Peer review is nothing but an extremely questionable method of publishing. Science needs to be criticized through rationality based on objective reality by nonscientists as well as scientists. To replace rationality with peer review is an extremely debased method of railroading fraud onto society.

Perhaps the ultimate, most basic, reason why the claim of peer review is a fraud is because no one can completely represent someone else's reality, and certainly not ill informed persons. It's like the pope determining morality for everyone else. How do you ask the pope if it is moral to cut trees for an oil well, or anything else in question? The persons who shout peer review can't spell the word science, and yet they speak for peer reviewed scientists.

Thanks for the additional.

Eris and the apple of discord *PING*.

Maybe your compatriots on TOS might like to spend some time on this...

Cheers!

Maybe your compatriots on TOS might like to spend some time on this...

Cheers!

Another "bash science" thread?

There seems to be a lot of those here lately.

A wide variety of quasi-scientific journals are available, and the mainstream scientific journals also publish letters that might cover topics not yet ready for prime time. For the mainstream journals of science, and of philosophy, the author should have his credentials in order or somehow get someone with creds to co-author. If someone lacks creds he also lacks the right to be heard by the scientific or philosophic community—he lacks standing.

No, just criticizing the peer review process as something of cronyism.

My doctoral advisor (who once said to me in earnest, "If I don't get a Nobel prize, there's no way Rudy Marcus will" -- about two years before Marcus got the phone call from Stockholm) got zinged by a reviewer who was even more prestigious than him stealing one of his ideas, down to the new nomenclature for the phenomenon.

So it isn't just wankers and wannabees who get shafted.

Cheers!

Another "bash science" thread?

If you don't mind me jumping in here; bashing science is actually the farthest thing from my mind. IMHO, science, and scientists applying the "scientific method", have been primary ingredients in the mix making America what it is today. America was the world leader in scientific and technological breakthroughs for the better part of 200 years. We're probably not the first; I'm sure the same thing could be said of past cultures and societies.

To the layman, it appears something has gone awry. That something seems to have had a negative effect, particularly in the areas of science. To the layman, what seems to pass for science these days is limited to theories and hypotheses that typically fit within a narrow field. Dogmatic adherence to "acceptable" science is what I see -- from the outside. "Acceptable science™" is where the money goes; anyone bucking the acceptable science™ system is on their own.

The NSF is requesting a budget of something approaching 7 billion dollars for 2009! Much of which will be awarded to acceptable™ projects. The weiners from academia who make up a large portion of the NSF board and the scientific community are naturally conjoined. The NSF board(academia weiners) are VERY likely fairly squishy liberals since that seems to be the makeup of academia these days. So, who gets the money? Squishy liberal members of the scientific community? It appears to this layman the system is heavily skewed in favor of liberal projects. THAT is at best, short sighted; at worst an agenda.

The NSF/NSB has apparently identified a problem re their adherence to acceptable science™ and have at least paid lip service to ADDRESSING THE ISSUE(a pdf file).

I know it's not your area of expertise, but just for grins would you humor me and give me your impression of THIS DISCUSSION, particularly as it relates to the expanding earth theory?

If someone lacks creds he also lacks the right to be heard by the scientific or philosophic community—he lacks standing.

Do tell. So, is it your belief standing is more important than potential breakthroughs - from whatever source? The amateur or lay types are at the mercy of the system™, and can only make contributions at the pleasure of the system™? The system then can, if it chooses, cherry pick what is good science? Do you see any potential for abuse here?

your belief standing is more important than potential breakthroughs

Let's just think a little about this. An incredible number of hoaxes, scams, errors and frauds await he who accepts every scheme that walks in the door. Be glad a filtering process exists at all. The noise level beyond the system makes every good idea simply one grain of sand on the beach; you would never find a good idea in the midst of all that chaos. We are already familiar with the popular 'breakthroughs.' It is remarkably easy to spot the bad ones. Standing is worth the trouble to get credentialed.

An incredible number of hoaxes, scams, errors and frauds await he who accepts every scheme that walks in the door.

Strawman. I would submit more than a few have made it through the process without so much as a discouraging word. They just happened to have the proper "tilt".

Be glad a filtering process exists at all. The noise level beyond the system makes every good idea simply one grain of sand on the beach; you would never find a good idea in the midst of all that chaos.

I'm not so sure about that. I think I would just as soon have all the noise and make up my own mind after the fact of "publishing". IOW, the dustup would occur in front of God and everybody instead of in the hallowed halls of academia or wherever they do these things. Maybe there are too many tender egos involved in the process that don't want to make fools of themselves in public. I dunno...

We are already familiar with the popular 'breakthroughs.' It is remarkably easy to spot the bad ones.

Sortof makes MY point doesn't it?

In any case, the web may end up taking a dump on the publishers' golden goose anyway; ANOTHER OPINION:

Getting published in the illustrious British scientific journal Nature is, frankly, a bitch. It's not just the years you spend designing the perfect experiment, or the hustling for grant money to collect the data. It's not even the long nights of trying to figure out how to express all that work elegantly in the cold language of scientific communication. No – the real trick is getting the editors at Nature to like it.

But that's still just the beginning: Those editors pick three or so relevant experts – from a list Nature requires you to submit – to anonymously assess your work's technical accuracy and overall merit. Those experts bounce it back to the editors, who add their own comments and send it to you asking for more work. If you decide it's worth the time and effort, you do it. And revise. And send it back to the reviewers. In the end, if everyone's satisfied, the article runs. If not, you submit it to another journal, one tier down, and do it all again. The process takes about four months.

That rigmarole is called peer review. Almost every journal does it, from marquee pubs like Nature to highly specialized periodicals like International Journal of Chemical Reactor Engineering. (No offense to IJCRE – you guys are a helluva read.) When it works, it's genius – quality control that ensures the best papers get into the appropriate pages, lubricating communication and debate. It's the quiet soul of the scientific method: After forming hypotheses, collecting data, and crunching numbers, you report the results to learned colleagues and ask, "What do you folks think?"

But science is done by humans, and humans occasionally screw up. They plagiarize, fake data, take incorrect readings. And when they do? Oy! Somebody always blames peer review. The process is lousy at policing research. Bad papers get published, and work that's merely competent (boring) or wildly speculative (maverick) often gets rejected, enforcing a plodding conservatism. It seems silly to say this about a system that's been in development since the mid-1700s, but the whole thing seems kind of antiquated. "Peer review was brilliant when distribution was a problem and you had to be selective about what you could publish," says Chris Surridge, managing editor of the online interdisciplinary journal PLoS ONE. But the Web has remapped the universe of scientific publishing – and as a result, peer review may finally get fixed.

The proof: In June, Nature began experimenting with a new method online. Authors submitting papers can choose a two-track process. While the work goes through the usual peer review drill, a preprint version gets posted on the Web. Anyone – even you – can comment, as long as you attach your name, affiliation, and email address. As of July, 25 articles had undergone this process, and the journal plans to issue a report late this year on how the test went. (Full disclosure: Wired editor in chief Chris Anderson participated in the project.) "The whole point of peer review is to help the editors select papers that are going to move science forward," says Linda Miller, US executive editor of Nature and the Nature research journals (Nature Biotechnology, Nature Genetics, et cetera). "If there's a better way, then why not? How could I say no?"

In other quarters, traditional peer review has already been abandoned. Physicists and mathematicians today mainly communicate via a Web site called arXiv. (The X is supposed to be the Greek letter chi; it's pronounced "archive." If you were a physicist, you'd find that hilarious.) Since 1991, arXiv has been allowing researchers to post prepublication papers for their colleagues to read. The online journal Biology Direct publishes any article for which the author can find three members of its editorial board to write reviews. (The journal also posts the reviews – author names attached.) And when PLoS ONE launches later this year, the papers on its site will have been evaluated only for technical merit – do the work right and acceptance is guaranteed. "Data becomes useful only if it's shared," Surridge says. "At the moment, our mechanisms for sharing information are the traditional journals, and if they're hard to get into, data is completely lost."

No one's sure which of these ideas, if any, will prevail. Sure, discarding anonymity will go a long way toward breaking up the old-boys' network, and open comments are great for nailing fakes and plagiarists. (The online community, not peer review, helped bust the South Korean stem cell fraud Woo Suk Hwang.) But Nature is an elite journal that accepts few submissions, a kind of exclusivity that lets universities use publication as a proxy for worth in hiring and promotion decisions. How can they assess papers published online and "reviewed" by an honors physics teacher? Have papers that went through an open process and got rejected been essentially published already? Plus, the idea of all these articles online, free for the Googling, terrifies the lucrative journal-publishing industry.

But seriously: Who cares? An up-and-coming researcher can get more attention from the right experts by publishing something earthshaking on arXiv than by pushing it through the usual channels. Crazy ideas will get batted around in moderated forums, which is pretty much what the Internet is for. Eventually, printed journal articles will be quaint artifacts. Scientific papers will be living documents with data published on Web pages – commented on, linked to, and mirrored by labs doing the same work 6,000 miles away. Every research effort will have thousands of reviewers working in real time. Today's undergrads have never thought about the world any differently – they've never functioned without IM and Wikipedia and arXiv, and they're going to demand different kinds of review for different kinds of papers. It's in their nature.– Adam Rogers

Fine. Read through the pile for yourself. Report back if your filtering technique comes up with anything.

Fine. Read through the pile for yourself. Report back if your filtering technique comes up with anything.

No offense RightWhale, but I would submit that MUCH of scientific and technological offerings can be easily understood by the average layman. The fly in the ointment is the language of the discipline being discussed. Sciencespeak, technospeak or whatever label you would like to apply to it is what separates the masses from the sciences. Now I'm sure this makes for lofty exchanges within the community™ where they are able to impress each other with their knowledge of the "language", which is probably not a bad thing under the circumstances.

Like so many other things it seems, science has become something that happens behind closed doors. Is it necessary, or even advisable? Laymen could care less about the language; give us the red meat.

I would really like to press on here, but gotta run for now.

For when you return: it is not at all the quality of the offerings but the volume of undistilled thought, half-baked or even raw ideas with zero possibility that keeps us from spending any moments of our short lives reading the stuff. Institutional publications have at least sorted out the somewhat coherent developments from the general muck.

Institutional publications have at least sorted out the somewhat coherent developments from the general muck.

Understood, and if you did have a quibble with the system and its apparent desire/need to defend sacred cows, what or where would it be?

Speaking of muck, how come I've never seen a program on Discovery, History or NG channels re those piles of bones you guys have stuck in the tundra up there? My God but that is some interesting stuff! But outside of the original discovery and book by Hibben(?), there has been little else to bring that anomaly, along with many others around the world, into the mainstream. You could become an instant star if you were able to get some teevee crews up there. Inquiring minds and all...

Piles of bones, and hides, too, are in the Museum at the Univ of Alaska. Even more piles of bones and teeth are in the Gruening Building at the Univ of Alaska where scholars are even now writing papers for peer-reviewed journals and doctoral theses. Maybe somebody could dramatize the bones for TV, but they are well-known locally since the gold mines sluiced away the muck.

...scholars are even now writing papers for peer-reviewed journals and doctoral theses.

On the case are they? Aren't they running about 60 years behind??? You don't suppose they will develop any theories about when and how the mountains of bones got there? Early Man made 'em do it??? Just curious, and since you're in the general vicinity, do you have an opinion?

Maybe somebody could dramatize the bones for TV, but they are well-known locally since the gold mines sluiced away the muck.

Yeah well, Alaska, in particular where the bones are, is not exactly in the top ten list of vacation spots, and their museums probably less so. So, just how many people will actually ever get any exposure to this amazing phenomenon?

Alaska, in particular where the bones are, is not exactly in the top ten list of vacation spots

By the planeload, year round.