同行评审（peer review）已成为现代科学出版的核心，以至于现在即使是最杰出的研究人员也必须经常经受其考验和折磨。

Over the years, this process has become so central to scientific publishing that nowadays even the most distinguished researchers must regularly subject themselves to its trials and tribulations.

不幸的是，同行评审的问题比任何特定审稿人员的素质问题还要严重。同行评审过程是不一致的和主观的，以至于——用英国医学杂志前编辑理查德·史密斯的话说——这“就象中彩票一样。”

Unfortunately, peer review has problems that run deeper than the quality of any particular reviewer. The process is inconsistent and subjective to the degree that — in the words of Richard Smith, a former editor of the British Medical Journal — it’s “something of a lottery.”

人工智能研究人员对2014年由具有影响力的神经信息处理系统会议组织者进行的一项实验结果感到震惊。两个不同的委员会对一部分提交的论文进行了评估，并作出接受或拒绝的独立决定。结果一个委员会接受的论文中有57%被另一个委员会拒绝。这与纯粹随机选择的结果非常接近。

Still, AI researchers were shocked by the results of an experiment conducted in 2014 by the organizers of the influential Conference on Neural Information Processing Systems. A portion of the submissions were evaluated by two different committees, which made independent decisions to accept or reject. It turned out that 57% of the papers accepted by one committee were rejected by the other. That’s unnervingly close to what you'd expect from purely random selection.

即使是最令人担忧的案例——明目张胆的错误或欺诈性论文——也很少被同行评议网发现。其中一个最令人震惊的例子是，德国物理学家简·亨德里克·肖恩（Jan Hendrik Schoen）在21世纪初发表了一系列据称具有开创性但实际上是欺诈性的论文。他被曝光的时候，那些试图在他工作的基础上进行研究的同事注意到，他的一篇论文中有重复的图片，导致发现更多的异常并最终进行全面的调查。在这之后，Schoen的数十篇经过仔细的同行评议的论文被撤回，其中包括在《科学》和《自然》两个最负盛名的期刊上发表的16篇令人瞠目结舌的论文。
Even the most alarming cases — papers that are blatantly wrong or fraudulent — are rarely caught in the peer review net. One of the most egregious examples is that of Jan Hendrik Schoen, a German physicist who published a slew of supposedly groundbreaking — but actually fraudulent — papers in the early 2000s. He was exposed when colleagues who were trying to build on his work noticed duplicated figures in one of his papers, leading to discoveries of additional anomalies and ultimately a full-blown investigation. In the aftermath, dozens of Schoen’s meticulously peer-reviewed papers were retracted, including an eye-popping total of 16 published in two of the most prestigious journals, Science and Nature.

肖恩丑闻主要起到警示作用，但它也暗示了尽管为什么科学事业能如此成功，同行评议还是存在缺陷。出版/发表只是一个更大的过程的一部分，在这个过程中，重要的论文被确定下来，然后由相关的科学群体进行严格审查。在当今的科学生态系统中，这一点在两方面都是真实的。像arXiv这样的在线预印本库使得论文在被提交出版之前就有可能获得广泛的声誉或名声。

The Schoen scandal mainly serves as a cautionary tale, but it also hints at why the scientific enterprise is so successful despite the shortcomings of peer review. Publication is just one part of a much larger process in which important papers are identified and then heavily scrutinized by the relevant scientific community. That’s doubly true in today’s scientific ecosystem, where online preprint repositories like arXiv make it possible for papers to achieve widespread fame or notoriety before they’re even submitted for publication.

一个科学家发表了多少论文，在哪里发表论文，这一问题在招聘、晋升、资助以及诸如计算机科学等学科甚至博士学位的授予等决策中起着巨大的作用。科学家的职业生涯可能取决于少数评审专家是接受还是拒绝一篇论文。

The question of how many papers a scientist published, and where, plays a huge role in decisions about hiring, promotion, funding and — in disciplines like computer science — even admission into Ph.D. programs. A scientist’s career may depend on whether a few reviewers choose to accept or reject a single paper.

--- ---

Peer Review Is Science’s Wheel of Misfortune

https://www.bloombergquint.com/gadfly/peer-review-is-science-s-wheel-of-misfortune

Peer Review Is Science’s Wheel of Misfortune

https://www.yahoo.com/finance/news/peer-review-science-wheel-misfortune-160036634.html