The review process should be transparent not confidential

A few days ago, I was alerted to a paper on Zelkova by Zhang et al., published in Tree Genetics & Genomes, because the authors cited our study from 2005 (Denk & Grimm 2005). Although, I’m out-of-business, I had to look at it (old habits), and I lost it. How could this pass the review process?
TGG is (well, was) a proper journal. I wrote a couple of mails, including one asking the responsible editor to clarify his decision (I will write a post on this, once the story evolved). And I got a swift answer (he was the only one who answered so far to one of my mails):

Dear Guido,
As you are well aware of the entire review process is strictly confidential, meaning that I can not comment on any part of it in response to a direct email. Instead I would like to direct you to the EiC of Tree Genetics and Genomes if you have any comments or if you have any formal complaints to file.
Sincerely,
-Pär Ingvarsson

There it was again, my old companion, the Impermeable Fog called review process confidentiality. Protecting the weak (editors and peers) from the wrath of the evil authors (and readers, in this case).

First, a graphic for those who have not published scientific papers to explain the battlefield known as single-blind peer review process.

Idealised scheme of the single-blind confidential review process

Second, let us list the pros and contras (well, depending on the viewpoint). The two main arguments for keeping the Impermeable Fog are:

Ensure that the peers can freely express their opinion on the submitted paper. This implies that peers have to fear retribution by the authors, when they criticise their work. Don’t think we can be mature enough to deal with criticism without going old-testament: You rejected my paper, I’ll turn down yours. Eye for an eye!
Spare unexperienced authors or non-native English-speakers the humiliation caused by the originally submitted version. We all (should) learn from our errors, and there should be no shame about it. Particularly, if you are unexperienced. Of course, we cannot expect that companies generating profits of 1 billion £ in six months can afford professional language editing services for non-Anglosaxon authors. (Those services charge around 15–25 € per page, quite a sum, compared to the 3000$ large publishing companies charge for open-access, and regarding that the usual paper has max. 15 pp...) But it is quite easy to recruit a retired Anglosaxon as co-author to clean the English of the paper, in case you can’t afford to pay the professional proof-reading service by yourself.

But there are more important arguments for keeping the Impermeable Fog:

Prominent co-authors do not have to take the responsibility (and blame) for a poorly designed and written paper submitted by their students or 2^nd or 3^rd world collaborators.
Peers can trash a work because of personal or science-political reasons. Peers know who the authors are, and they know who the editor is (see graphic). So, an important person can easily flex his (rarely her) muscles to force the editor into a decision. Or, in the words of Jim Reiding, editor of the American journal Palynology: ‘One very well-qualified researcher [who stayed anonymous] asked for “reject and resubmit”. I must abide by that.’ (Two other anonymous peers were very positive, and a third, also not an un-prominent person in the field who signed his review called for a major revision.)
Editors don’t have to take any responsibility for the papers they have deemed publishable. They can also choose not-really-experts or hopeful fresh future authors to act as peers, when the actual experts are currently unavailable for (unwilling to) review.
Peers and editors can also pack together to avoid heretic or critical ideas being published. You may remember #parsimonygate, which, actually, was no surprise at all (see also this post). There are probably worse cases, which we don’t know about because the review process is confidential to protect us from the evil in the (scientific) world.
Authors, peers, and editors can establish syndicates, transforming the ‘peer review’ into a ‘pal review’ process (apparently something providing a loop-hole for the double-blind peer review).

[PS In case you have stories about what happened in the Impermeable Fog, but need to remain anonymous, just send me an email; I’m putting a list together of things that simply should not happen during review.]

No matter in which context: When you cloud something, it becomes shady. And corruption spreads, because third parties cannot assess, where it starts and where it ends. Frustration and cynism eats its way through the community. But the only thing surfacing are papers published in established, peer-reviewed journals that should not be there while (much) better ones have been rejected and end up in regional no-impact journals, and you have no clue why (well, you may have some quite good ideas, but no proof thanks to the Fog).

But what are the alternatives? Open review, where you know your peers, or free review, where anyone of the community is invited to comment, don't work (at least, not outside rockbottom-hard fields of science, where everything can be tested and proved). Too many scientists are, and sadly have to be, too afraid of exposing themselves as critics and stating an honest opinion.

But there is an alternative that seems to work just fine and should be the rule.

Just make the peer review process transparent by publishing the original draft, the peer reviews, the authors’ rebuttal letter(s), and the editor’s decision letter(s).

One journal that is fully dedicated to transparency (and increasingly successful), including the review process and without forcing authors and reviewers exposing themselves to perceived or real threats of retribution, is PeerJ. They leave it to the authors to decide on show-all or hide how the paper came into existence (they recommend to show; and many do). I published most of my last professional papers there.
I was shocked, how consistently constructive and careful reviews can be, even from peers who seem to be quite sceptic about your work (if I made a PeerJ-like experience 10 years ago, I possibly would still be a professional scientist). Traditionally, the papers that I was involved in and got dumped into the Impermeable Fog had about 25% constructive (or plain positive) reviews; the rest was something in between pure proofing (language naturally, structure, format) or general nagging on the one end, and utterly incompetent, inacceptable, self-centred sermons on the other (I plan to post a best-of, bite me Fog!).
Also, all editors of PeerJ handling our papers seemed to be really in it, no matter whether the topic was a tiny curiosity (Grímsson, Grimm & Zetter 2017; the paper Palynology could not publish because the editor had to abide to the anonymous expert, and whose critiques we simply ignored when revising the paper for PeerJ) or something that may change an entire field of science (Bomfleur, Grimm & McLoughlin 2017); reviews, editor's letter, and response include some very educative points for researchers in the touched fields, so take a look.
The latter is a priceless boon of transparent peer review. The often educative discussion between authors, peers, and editors does not dissipate into the obscurity of the Impermeable Fog. Unexperienced researchers can follow the mechanics of the peer review process of similar papers. Ideally, helping them to avoid the same errors that we made. Thus, it potentially enhances the quality of submissions, and reduces the workload for peers and editors.

Other journals practising review process transparency are e.g. The EMBO Journal (see also this article in Nature) or some of the journals published by Copernicus Publishers on behalf of the European Geosciences Union such as Biogeosciences or Climates of the Past.

But, the most striking argument for review transparency is:

Contributions of the peers and editors are clearly visible.

And this is well deserved and long overdue.

Fact is: Whether acting as peer or editor, the Impermeable Fog prevents honest gains from participating in the review process (except occassional personal enlightment).
[Info for non-scientists: Neither authors, nor peers, nor editors are actually paid for their work on publications; they do this because they (as authors) have to, to get an elongation, grants, and eventually a permanent position, or (as peers and editors) want or feel obliged to because they are idealistic or for one of the shadier reasons outlined above (see example in my introductory post). Most of them could make much better use of their time.]

For instance, when, as peer, I make a half-ready paper publishable and reveal myself (which some journals explicitly prohibit “to protect their peers and objectivity of the review process”), the best I can hope for is a place in the acknowledgements. Possibly in the same line with the other peer, who did nothing. Worst case: with the nagging feeling that there are co-authors on the paper who apparently did not spend half-a-day thinking about this particular study (because if they did, I would not have needed to spend so much time on the review). It can also happen that I spend days writing a very careful, but critical review, but the second peer says “publish as-is” – for whatever reason – and the editor lets the authors follow that opinion – for whatever reason. The Impermeable Fog will cover this, and as a peer, you're bound by confidentiality and incapable of doing anything against this.

As editor, I am in a permanent quandary. It can take a lot of effort to find peers willing to provide professional and timely reports. I may have to settle with some very suboptimal choices in the end (or bend the rules), while the authors are desperately awaiting a decision because they need the paper out. I also may have to keep an eye on the possible impact (e.g. to keep the journal at an IF > 2). And I do all this in addition to my full-time job as a scientist. I may even find myself in the situation that I find a paper quite good, but the (expert) reviewer(s) are destroying it (this happened to several editors of my papers, apologies), ending-up trying to find loop-holes to accept the paper despite journal regulations. [Still grateful to Edith Zimmer for her handling of the, back then quite heretic, paper on Isothecium (Draper, Hedenäs & Grimm 2007), which got a quick bad review, which she effectively just ignored and told us to do the same, and when she got sick waiting months for the needed second, she did it herself and published our paper, in some violation of journal policy; 48 citations so far show that this was not a bad decision for the impact factor of the journal.]

Review documents of PeerJ, Biogeosciences, and Climates of the Past have their own permanent doi, hence, they can be cited. PeerJ further indirectly awards peers. Authors can pay once for a lifetime of open-access publishing, but the membership gets frozen (de-frost fee is 99$), when you are not giving back once a year by acting as a peer, engaging in a Q&A, or an online discussion

The transparent review process ensures fair treatment of authors and can help to educate and guide young researchers, it prevents corruption, and allows peers and editors to take credit for their work. It is a win-cube situation!

So why are there still so very many journals using a non-transparent single-blind review bound by confidentiality?

Because the public, the non-scientists, are not bothered by the fact that their money (see graphic above) is used to keep up a shady system. And as in any other shady system, there are victims and profiteers.
It’d be easy to blame authors, peers, and editors to make the most out of the opportunities provided by the Impermeable Fog. Why don’t we just boycott journals that do not enact transparent review processes? Because, scientists need papers in journals with as high as possible impact to get jobs and research money, and this means many are somewhat limited in their choice, where to publish during phases of their career. You play by the rules of the Impermeable Fog or you can forget about publishing in those journals. Decline the invitation of an editor, who was your supervisor, may be your next peer or even co-operation partner? Bad idea, unless you can afford to burn those bridges, too. And sometimes, you need to become an editor of an established, no matter how foggy, journal for your next career level. Plus, not all of us are working on an ethical basis. The Co-existence Approach is just one example for a method that would not have cornered an unspoiled market when the review process would be transparent.

Very few scientists can afford to stand up openly against the system. And no-one can tell how many of those who could afford to stand up, don’t want to, because it’s so cosy in the Fog.
The change has to be forced from the outside: research agencies and state-funded research institutions, who generate most of the profit of private companies publishing science, should force those publishers to promote actual transparency of the review process (but see e.g. Elsevier’s understanding of transparency).
No exceptions (this includes you, Science and Nature, may your sky-rocking impact factors rest in peace). And the greater public should take interest that they do so, it’s their money!