Posted on 07/17/2021 2:42:21 PM PDT by karpov
I recently submitted a manuscript to an education journal, a review essay of another scholar’s work. It opened with a compliment of the author’s “highly-praised and influential work.” To that statement, one reviewer of my manuscript asserted that I used “emotionally loaded language of incredulity, dismissiveness, and hyperbole.”
More “tone policing” comments riddled the review, suggesting that even when my words might sound benign or complimentary, what I really meant was malevolent. There were several examples of another curious critique type as well: Raw declarations that my claims could not possibly be true, without any effort having been made to follow my footnotes to the evidence.
Few of my manuscript submissions to education journals have been reviewed substantively. For those familiar with Paul Graham’s hierarchy of disagreement, the reviews tend to lack a refutation of the central point, refutation in general, or counterargument. Instead, most consist of responding to (perceived) tone or being ad hominem (even anonymously) attacks.
I consider it likely that this reviewer knew that I had written the manuscript. And that may have motivated the decision to volunteer. If I read the editor’s reviewer numbering system correctly, three others had agreed to review but then failed to follow through. Who knows how many were originally asked?
As it is, the traditional peer review system relies on unpaid volunteers from a small population of very busy people to perform an intensive and time-consuming task. Two types of scholars feel most compelled to review papers: Those intrinsically motivated (not necessarily for noble reasons) such as the aforementioned reviewer, and those who “have to,” such as graduate students and not-yet-tenured professors.
(Excerpt) Read more at jamesgmartin.center ...
If your peers are, as in the case of various iterations of critical theory, a bunch of raving Marxist loons, all peer review will accomplish is increasing the crazy.
Just saying.
A standard is only as good as the enforcement mechanism.
So if you peer review a study, and fail it, they will do the same to you. No one really reads them, so just punt and approve.
I would like to know exactly what it was that he reviewed and what he had to say about it. It sounds like those reviewing the review were protecting something.
Science is fact.
Scientists are corrupt thugs.
The author of this article is too kind in his criticisms. I have seen reviews that are utterly vicious and filled with inaccuracies. I recently read four reviews of articles submitted to a ranked journal. One reviewer made no comments, only filled out a brief survey. Another reviewer, in limited English, made claims about the article that were simply untrue. It appeared to be a semi-boilerplate review the reviewer may submit for most papers they review. This gets them credit for reviews when they submit their activity reports to their school. Very unethical.
Another reviewer wrote about the “poor writing” regarding a paper that I knew was proofed and scrutinized by top academics and one professional editor. In contrast, the reviewer’s writing was seriously flawed.
Across many reviews I note some repeated flaws. The reviewers tend not to understand the research methodology, which can require sophisticated knowledge. Usually, the reviewer just blames the author for not explaining the method. This is often used to cover up the reviewer’s ignorance. Another popular reviewer flaw is to criticize the lack of recent citations. I have actually read a review that criticized the author for not including 2021 citations when the paper was submitted only two months into 2021—ridiculous. This conflicts with the important requirement that the originator of a concept must be cited—that usually means at least one citation must be many many years old if the topic is of much consequence.
There are many flaws among reviewers. I conduct many reviews myself and there are journals with which I will not cooperate because of the poor reviewers—which means the editors are lazy or poor themselves. When you see bad reviews, you know there are bad editors. Another editor flaw is to expect citations to them or their journal. If the journal or the editor publishes useful work, it will get cited. An editor should never demand citations but it happens.
This is a great article that points out some of the weaknesses of the “gold standard” for publishing research.
I publish in math journals and maybe because the subject is less political than most, the reviewing system seems to work pretty well. Reviewers are primarily concerned with, is it correct, is it new, and is it interesting, with only the third question being subjective.
In our company we have many times attempted to replicate patents or publications to develop new products. Some of the claims by scientists are spot on, but often other scientists’ work cannot be duplicated.
The gorilla in the corner that nobody mentions.
The reviewer has “X” number of years invested in a competing theory that you’ve just disproved. They are chosen/asked to review the article because of their supposed expertise. It is easier to torpedo somebody else’s work than admit you’ve wasted years on something untrue.
Rd later.
Exile marxists.
That is good to know. Those criteria you identify—correctness, newness, and interesting nature—seem suitable to some other disciplines as well. Despite that, the quality of the reviews is too often poor. Obvious errors and flawed interpretations. Thanks for your comment!
I suspect that happens, especially when your work could render the reviewer’s work obsolete. Thanks for your comment!
This does not surprise me at all.
Peer review is just the good old boys club patting each other on the back and pulling favors for each other.
Not part of the club?
Too bad for you.
I don’t know about educational journals but in engineering journals all personal and emotional references are avoided. Even complimentary ones.
Very often it is a good old boys club and all criticism of established authors are avoided. It is common practice to avoid publishing any article newsworthy until you have been around for awhile and been accepted into the club.
If you compare currently literary style to pre - 70’s you will realized a dramatic decrease in quality and ability. Today’s is more like “Run spot run”.
Because my mentor was not part of the club I did not learn the politics early on. For six years I was not able to get published in the US as sole author. Within a year after quitting the lab, six papers were published in a highly regarded international journal. Some of them provided ample evidence the opinions of the established “experts” in the US were incorrect. I should have first published worthless articles to get established.
The gold standard sometimes works, but don’t count on it.
Bookmarked. Later.
It happened to my wife who was a big name in Special Education in Asia.
I am sorry your wife suffered through this. I only scratched the surface of the garbage that appears in journal reviews and editing. I am glad this article has appeared and hope more is written about this problem in the future.
Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.