Can offering choice to researchers reduce researcher bias?

High-quality peer review has never been more important to validate the science we publish. With research integrity regularly hitting the headlines, especially since the COVID pandemic, publishers and reviewers play a pivotal role in ensuring robust, reliable research gets published, and flawed works do not.

The backbone for upholding integrity is peer review, where experts in the same field assess the quality of research. The process is designed to safeguard high standards, help improve promising work and weed out problematic papers, but there are some flaws in the process.

Peer review flaws

A well-documented issue is bias in peer review. Whether conscious or otherwise, it compromises fair judgment based on things like gender, name, nationality, affiliation, or career status. To mitigate against this, new peer review approaches are introduced and tested for their efficacy.

One such approach is double-anonymous peer review, where the identities of authors as well as reviewers are concealed. The Institute of Physics was the first STM publisher to offer double-anonymous peer review across all our propriety journals. The move is part of our dedication to tackle inequality in the scholarly publishing process. We have chosen to implement double-anonymous on a voluntary basis to give our authors the choice and to help us examine the efficacy of this approach as a tool against bias. We hope that authors recognize the benefits of research being judged on merit rather than factors such as the prestige of the institution they work for.

The initial results are encouraging, indeed recently Nobel laureate Novoselov anonymized his manuscript when submitting his work to one of our journals, demonstrating a belief in the publishing system and a trust in the quality of the research rather than relying on their established reputation. But the question is, if we don’t enforce double-anonymous and simply encourage it, do we see a reduction in peer review bias at all? Is there even a bias against double-anonymous papers, with reviewers assuming the author may be hiding their identity for a reason? Putting this question to the test in publishing, we partnered with researchers from the University of Michigan, providing data on our journey to double-anonymous peer review.

Leveling the playing field

The researchers looked at the submissions from a large number of researchers—390,000 authors and 168,000 reviewers from around the globe. The outcome of the study is encouraging and shows that offering double-anonymous peer review increases the likelihood of positive reviewer recommendations for low-prestige authors by 2.4% and lowered it for middle- and high-citation authors by 1.8% and 1%, respectively.

But the most exciting finding was the fact that the policy has the biggest effects on reducing prestige bias on final paper decisions, increasing acceptance of low-prestige authors by 5.6% while lowering it by 4.6% and 2.2% for middle- and high-citations authors, respectively. In short, double-anonymous peer review levels the playing field. We are yet to formally study the impact on other types of bias, e.g. gender and geography, although the initial results look intriguing, with researchers from some parts of the world being twice as likely to have their work accepted under the double-anonymous method.

Offering researchers a choice rather than forcing them delivers some of the benefits of hard-to-implement policies at a tremendously lower cost. And even though the results are encouraging, we believe that double-anonymous peer review is just one method in support of greater integrity of research. As a sector, we need to work together to evolve best practices and support and educate researchers in their reasoning and the consequences of their publishing choices.