Transparent anonymous peer review policy

Some best publications in the most active realm of research (i.e. in computer science, see examples below) already practice different versions of the follwing transparent anonymous peer review policy. The reviews are publicly available together with specific versions of papers to which the reviews are written (on arxiv or elsewhere). The reviews are anonymous unless the reviewer wants to sign his/her review.

Examples of transparent peer review

  • Electronic Proceedings of the Neural Information Processing Systems Conference
  • International Conference on Learning Representations, see Reviewing Process.
  • Open review platform used by some conferences (=journals) on computer science.
  • Collaborative Peer Review.
  • Moscow Mathematical Conference of High School Students (the page where reports are published, mostly in Russian: рецензирование работ на Московской математической конференции школьников).

    Relevant examples

  • Review the peer review process.
  • Pub Peer: online platform for post-publication peer review.
  • Online platform for pre- and post-publication peer review in mathematics (topology of manifolds). Example.
  • Incompetent handling of a paper in `Letters in Mathematical Physics'.
  • Using peer review system to promote unreasonable opinions which do not stand open discussion: \S2 (more striking examples to appear in the update), Remark 4.1.
  • Incompetent handling of an expository paper in "Summer School `Modern Mathematics'" series (in Russian).

    Discussion

  • It is now hard to convince people of high scientific level without high level of transparency.
  • We must demand evidence of peer review.
  • Make reviews public, says peer review expert.
  • Can I publish the reviews I receive? Can I publish the reviews I write?
    Researchers have a duty to expedite the scientific process. A publicly-funded scientific process should be transparent. Authors, reviewers and editors should all be accountable for their actions. There are now more formal options for sharing your reviews; for example via Publons. [ChrisSampson87]
    ... I totally support the sharing of provided peer reviews after publication. I believe transparency can only add to speed and quality of scientific research, and most of the major flaws in peer review rely on the fact that it is kept secret behind curtains by most of the community. Likewise I am in favor of signing my reviews -- this is not always easy, and often leads to a backlash, which is halfway gone towards sharing it later. When one signs a review and plans on sharing it later, immediately a greater sense of responsibility is created. There are websites allowing for this practice which implies that any member journals include editors who are OK with the practice, and authors submitting papers to those journals should be aware of the possibility of greater exposure after their publication. Minor personal conflicts and egoistic feelings aside, I strongly believe bringing in more light to peer reviews, even of rejected papers, would do far greater good than damage to the working conditions of scientists everywhere. [Scientist]
  • This is an interesting idea. Personally I would not mind publicizing my reviews anonymously. My recent impression about the peer reviews of my own papers is that not everything is OK in the commonly used scheme. In most cases the reviews I receive (both positive and negative) show that the reviewer did not read the paper. In some cases the reviews even refer to the content of the paper incorrectly. As you might guess, all such review took months or even years to be written, possibly to indicate that the formalities are properly met. [A reply from a colleague.]
  • Принятие решений без ответственности за них - в частности, без информирования публики, на основании каких документов решение принято - коррупционная практика. В частности, закрытость рецензирования в научных журналах нередко приводит к некомпетентности - в отзывах, а тогда и в принимаемых решениях, а тогда и в управлении наукой. Нередки попытки использовать закрытость для подковерного продвижения тех мнений, которые невозможно отстоять в открытой научной дискуссии - см., например, примеры выше.
  • My sympathies are certainly with all victims of abuse provoked by a non-transparent reviews system. The question is what you lose when you change this system. You can make all reviews public, and even not anonymous (publicity in itself obviously reduces the degree of anonymity, since there are usually a lot of people familiar with the reviewer's style, and the reviewer may well want to hide himself from one of them). This naturally makes reviews less frank and explicit. For an editor, this makes the work more difficult. (F. Petrov)

    [ Last modification 28.11.2021. Please send comments to Arkadiy Skopenkov, s*open*o@mccme.ru, where *=k. ]