Double-anonymous peer review reduces reviewer bias, finds a three-year trial

The British Ecological Society has published the results of a three-year randomized trial comparing double and single-anonymous peer review. The findings indicate a reduction in reviewer bias when author identities are anonymized.

The three-year randomized trial, published in the journal Functional Ecology, provides the most comprehensive data yet on the effects of anonymizing authors during scholarly journal peer review.

Double-anonymous peer review, also referred to as double-blind peer review, is where author identities are not disclosed to reviewers. This differs from single-anonymous peer review, where reviewers know the identities of the authors, which is more traditional in life science disciplines.

The study found that when author identities were anonymized and reviewers did not know whose paper they were reviewing (double-anonymous review), peer review outcomes were similar across author demographics.

In contrast, when author identities were known by reviewers (single-anonymous review), outcomes depended on author demographics. Papers with a first author residing in a higher-income country or a country with a higher average English-proficiency received higher ratings from reviewers and were more likely to be invited for revision or resubmission than papers with first authors from a lower income country or country with a lower average English-proficiency.

Interestingly, anonymizing author identities had no effect on gender differences in reviewer ratings or editor decisions.

Professor Charles Fox of the University of Kentucky, the lead author of the study and previously the Executive Editor of Functional Ecology said, “Our trial provides strong evidence that authors from higher-income and/or English-speaking countries receive significant benefits to being identified to reviewers during the peer review process, and that anonymizing author-identities (e.g., double-blind review) reduces this bias, making the peer review process more equitable.

“It’s critical for science, and for the scientists involved, that peer review be a fair and unbiased process. The results of this trial will help inform publishers on the best ways to minimize some sources of bias in the publishing process.”

Based on the strong evidence provided from the trial, the British Ecological Society will begin transitioning its journals to mandatory double-anonymous peer review. This will begin with Functional Ecology, with the other journals published by the BES to follow.

Andrea Baier, Director of Publishing at the British Ecological Society, said, “The British Ecological Society is committed to promoting equitable practices in international ecological science. Authors from all over the world submit to our seven journals and it is vitally important that the research we publish is reviewed and selected in the most impartial way, regardless of the authors’ backgrounds.”

Professor Rob Freckleton, University of Sheffield and Chair of the British Ecological Society’s publications committee, said, “On behalf of the British Ecological Society, the publications committee supported this important experiment, and from the outset we committed to being led by the results it would produce. We now have the evidence that double-anonymous peer review is an important building block towards greater equity in publishing and we are acting on it.”

About the trial

To explore how blinding author identities during peer review reduces biases based on author location or gender, Functional Ecology conducted a large, randomized trial, using real manuscripts submitted from 2019 to 2022. Every research paper submitted to the journal (3,739 submissions, 1,432 sent for peer review) was randomly assigned to be reviewed double-anonymous or single-anonymous.

Authors were required to submit their papers to be reviewed with their identities anonymized. Half of submissions were then randomly chosen to have author details added to the papers by adding a title page with information that identified the authors.