经典句子： 

 specialist classes make informational gatekeeping processes into safeguards for their own status.

专业阶层将信息把关过程变成了保护他们自身地位的保障。

======

https://blog.sciencenet.cn/blog-279293-1377383.html

https://www.experimental-history.com/p/the-rise-and-fall-of-peer-review

The rise and fall of peer review

同行评审的前因后果

美国学者：同行评议失败已成定局，应全盘抛弃！|学术界|学术论文|审稿人|科学_手机网易网 (163.com)

=========

英汉对照

Why the greatest scientific experiment in history failed, and why that's a great thing

为什么同行评审这场历史上最伟大的科学实验失败了，为什么它的失败是一件大好事。

ADAM MASTROIANNI，2022年12月14日

For the last 60 years or so, science has been running an experiment on itself. The experimental design wasn’t great; there was no randomization and no control group. Nobody was in charge, exactly, and nobody was really taking consistent measurements. And yet it was the most massive experiment ever run, and it included every scientist on Earth.

在过去60年的时间里，科学一直进行同行评审实验，涉及科学本身是否科学的实验。该实验的设计并不好;没有随机分组，也没有对照组。没有人对这项实验负责，也没有人真正评估这项实验的结果。然而，这是有史以来规模最大的实验，它囊括了地球上的每一位科学家。

Most of those folks didn’t even realize they were in an experiment. Many of them, including me, weren’t born when the experiment started. If we had noticed what was going on, maybe we would have demanded a basic level of scientific rigor. Maybe nobody objected because the hypothesis seemed so obviously true: science will be better off if we have someone check every paper and reject the ones that don’t pass muster. They called it “peer review.”

大多数人甚至没有意识到他们参与了实验。他们中的许多人，包括我自己，在实验开始时还没有出生。如果我们注意到发生的一切，也许我们会要求实验从一开始就要有基本的科学严谨性。但是没有人反对这项实验，可能因为这个实验的假设似乎是如此明显地正确:如果有人检查每一篇论文，并拒绝那些不合格的论文，似乎科学应该变得更好。这件事他们称之为“同行评议”。

This was a massive change. From antiquity to modernity, scientists wrote letters and circulated monographs, and the main barriers stopping them from communicating their findings were the cost of paper, postage, or a printing press, or on rare occasions, the cost of a visit from the Catholic Church. Scientific journals appeared in the 1600s, but they operated more like magazines or newsletters, and their processes of picking articles ranged from “we print whatever we get” to “the editor asks his friend what he thinks” to “the whole society votes.” Sometimes journals couldn’t get enough papers to publish, so editors had to go around begging their friends to submit manuscripts, or fill the space themselves. Scientific publishing remained a hodgepodge for centuries. (Only one of Einstein’s papers was ever peer-reviewed, by the way, and he was so surprised and upset that he published his paper in a different journal instead.)

这项实验带来了巨大的变化。从古代到现代，科学家们都是通过书信交流思想。阻碍他们思想交流的主要障碍只是纸张、邮资、印刷的成本。在极少数情况下访问天主教会就能解决费用问题。科学期刊在17世纪出现时，它们的运作方式更像是杂志或通讯。它们挑选文章的过程从“得到什么就印什么”到“编辑通过咨询他的朋友而后选择”再到“学会成员投票”。有时期刊无法获得足够的论文来发表，所以编辑们不得不四处乞求他们的朋友投稿，或者自己写点东西填补版面。几个世纪以来，科学出版一直是一个大杂烩。（顺便说一下，爱因斯坦的论文中只有一篇经历了同行评议，结果使他非常惊讶和沮丧，最终结果是：他的论文发表在了另一本期刊上）

That all changed after World War II. Governments poured funding into research, and they convened “peer reviewers” to ensure they weren’t wasting their money on foolish proposals. That funding turned into a deluge of papers, and journals that previously struggled to fill their pages now struggled to pick which articles to print. Reviewing papers before publication, which was “quite rare” until the 1960s, became much more common. Then it became universal.

第二次世界大战后，一切都改变了。政府给研究注入资金，他们召集“同行评审专家”，以确保他们不会把钱浪费在愚蠢的研究方案上。这些资金变成了蜂拥的论文，以前难以填满版面的期刊现在很难选择要发表哪些文章。在20世纪60年代之前论文发表前进行评审是“相当罕见的”，然后变得常见，之后全面铺开。

Now pretty much every journal uses outside experts to vet papers, and papers that don’t please reviewers get rejected. You can still write to your friends about your findings, but hiring committees and grant agencies act as if the only science that exists is the stuff published in peer-reviewed journals. This is the grand experiment we’ve been running for six decades. The results are in. It failed.

现在，几乎所有的期刊都使用外部专家来评审论文，而那些审稿人不喜欢的论文就会被拒稿。你虽然可以写信告诉你的朋友你的发现，但招聘委员会和资助机构只把发表在同行评议期刊上的东西唯一地当作科学。这就是我们进行了60年的伟大实验。现在结果出来了，完全失败的结果。

A WHOLE LOTTA MONEY FOR NOTHIN’

Peer review was a huge, expensive intervention. By one estimate, scientists collectively spend 15,000 years reviewing papers every year. It can take months or years for a paper to wind its way through the review system, which is a big chunk of time when people are trying to do things like cure cancer and stop climate change. And universities fork over millions for access to peer-reviewed journals, even though much of the research is taxpayer-funded, and none of that money goes to the authors or the reviewers.

投了一大笔钱，却一无所获

同行评议是一项巨大而昂贵的干预。据估计，科学家每年总共要花15000年的时间来审阅论文。一篇论文可能需要几个月或几年的时间才能通过评审，这相当于人们试图做治愈癌症和阻止气候变化等花费的时间，这是一大块时间。大学为获得同行评议的期刊的阅读权要支付数百万美元，尽管很多这些研究是由纳税人资助的，而且这些钱没有一分钱给作者或审稿人。

Huge interventions should have huge effects. If you drop $100 million on a school system, for instance, hopefully it will be clear in the end that you made students better off. If you show up a few years later and you’re like, “hey so how did my $100 million help this school system” and everybody’s like “uhh well we’re not sure it actually did anything and also we’re all really mad at you now,” you’d be really upset and embarrassed. Similarly, if peer review improved science, that should be pretty obvious, and we should be pretty upset and embarrassed if it didn’t.

巨大的干预应该产生巨大的影响。例如，如果你在一个学校投入1亿美元，结果很可能让学生们获益。几年后如果你问，“嘿，我的1亿美元对学校系统有什么帮助吗?”而每个人都说，“嗯，我们不确定它到底起到什么作用，而且我们现在对学校很不满意”。你会非常沮丧和尴尬。同样，如果同行评议促进了科学，那应该是很明显的，如果没有，我们应该感到非常沮丧和尴尬。

It didn’t. In all sorts of different fields, research productivity has been flat or declining for decades, and peer review doesn’t seem to have changed that trend. New ideas are failing to displace older ones. Many peer-reviewed findings don’t replicate, and most of them may be straight-up false. When you ask scientists to rate 20th century discoveries in physics, medicine, and chemistry that won Nobel Prizes, they say the ones that came out before peer review are just as good or even better than the ones that came out afterward. In fact, you can’t even ask them to rate the Nobel Prize-winning discoveries from the 1990s and 2000s because there aren’t enough of them.

同行评议没有效果。在各种不同的领域，研究效率几十年来一直持平或下降，同行评议似乎并没有改变这一趋势。新思想不能取代旧思想。许多同行评议的文章结果无法重现，其中大多数可能是完全错误的（译者注：90%的CNS文章都是错的）。当你让科学家评价20世纪的物理学、医学和化学的诺贝尔奖发现时，他们会说，在同行评议之前出现的发现比在同行评议之后出现的发现更好。因为，你甚至不能让他们对20世纪90年代和21世纪初授予的诺贝尔奖进行评价，因为这个时期并没有多少重要的发现。

Of course, a lot of other stuff has changed since World War II. We did a terrible job running this experiment, so it’s all confounded. All we can say from these big trends is that we have no idea whether peer review helped, it might have hurt, it cost a ton, and the current state of the scientific literature is pretty abysmal. In this biz, we call this a total flop.

当然，自第二次世界大战以来，很多其他东西都发生了很糟糕的变化。我们的同行评审实验做得很糟糕，实验很混乱。从变化的大趋势看，我们不知道同行评议是否有帮助，它很阻碍了科学的进步，而且成本很高，而且目前的科学文献状况相当糟糕。同行评审，可以说是彻底失败了。

POSTMORTEM

What went wrong?

Here’s a simple question: does peer review actually do the thing it’s supposed to do? Does it catch bad research and prevent it from being published?

实验结果

同行评审的问题是什么?

这里有一个简单的问题:同行评议真的做到它被假定应该做的事情吗?它能阻止不好的研究发表吗?

It doesn’t. Scientists have run studies where they deliberately add errors to papers, send them out to reviewers, and simply count how many errors the reviewers catch. Reviewers are pretty awful at this. In this study reviewers caught 30% of the major flaws, in this study they caught 25%, and in this study they caught 29%. These were critical issues, like “the paper claims to be a randomized controlled trial but it isn’t” and “when you look at the graphs, it’s pretty clear there’s no effect” and “the authors draw conclusions that are totally unsupported by the data.” Reviewers mostly didn’t notice.

答案是否定的。科学家们做过一些研究，他们故意在论文中添加错误，把论文发给审稿人，然后考察审稿人发现了多少错误。审稿人的成绩非常糟糕。在这写研究中，审稿人发现了30%的主要错误、他们发现了25%、他们发现了29%。这些都是错误都是关于关键性的问题，比如“这篇论文声称是一项随机对照试验，但事实并非如此”，“根据图表中的数据，很明显没有效果”，即“作者得出的结论完全没有数据支持”。但是审稿人大多没有发现。

In fact, we’ve got knock-down, real-world data that peer review doesn’t work: fraudulent papers get published all the time. If reviewers were doing their job, we’d hear lots of stories like “Professor Cornelius von Fraud was fired today after trying to submit a fake paper to a scientific journal.” But we never hear stories like that. Instead, pretty much every story about fraud begins with the paper passing review and being published. Only later does some good Samaritan—often someone in the author’s own lab!—notice something weird and decide to investigate. That’s what happened with this this paper about dishonesty that clearly has fake data (ironic), these guys who have published dozens or even hundreds of fraudulent papers, and this debacle:

事实上，我们已经得到了彻底说明同行评议没有用的真正数据:欺诈性论文一直在发表。如果同行评审有效，我们应该听到的故事是:“科尼利厄斯·冯·弗拉克教授今天因为试图向科学期刊提交一篇造假论文而被解雇了。”但我们从没听过这样的故事。相反，几乎每一个论文不端都是从论文通过评审并发表而后被撤稿。论文发表之后才有好人——通常是作者自己实验室里的人! ——注意到一些奇怪的事情并决定追查。结果发现，这篇显然有虚假数据的不诚实论文 (讽刺)的作者还发表了几十甚至几百篇欺诈性论文——崩溃！

Wait a second, these are not real error bars … the author literally just put the letter “T” above the bar graphs

图中的并不是误差条，作者只是把字母T放在曲线上。

Why don’t reviewers catch basic errors and blatant fraud? One reason is that they almost never look at the data behind the papers they review, which is exactly where the errors and fraud are most likely to be. In fact, most journals don’t require you to make your data public at all. You’re supposed to provide them “on request,” but most people don’t. That’s how we’ve ended up in sitcom-esque situations like ~20% of genetics papers having totally useless data because Excel autocorrected the names of genes into months and years. (When one editor started asking authors to add their raw data after they submitted a paper to his journal, half of them declined and retracted their submissions. This suggests, in the editor’s words, “a possibility that the raw data did not exist from the beginning.”)

为什么审稿人不能发现基本的错误和公然的欺诈?一个原因是，他们几乎从不查看他们审查的论文背后的数据，而这正是最可能出现错误和欺诈的地方。事实上，大多数期刊根本不要求你公开你的数据。你应该“应要求”提供这些信息，但大多数人拒绝不提供。这导致最终会出现的剧情，比如大约20%的遗传学论文提供的数据完全无效——Excel会自动将基因名称更正为月份和年份。(当一位编辑开始要求作者在他的期刊上提交论文后添加原始数据时，一半的人拒绝并撤回了他们的投稿。用编辑的话来说，这表明“原始数据可能从一开始就不存在。”)

The invention of peer review may have even encouraged bad research. If you try to publish a paper showing that, say, watching puppy videos makes people donate more to charity, and Reviewer 2 says “I will only be impressed if this works for cat videos as well,” you are under extreme pressure to make a cat video study work. Maybe you fudge the numbers a bit, or toss out a few outliers, or test a bunch of cat videos until you find one that works and then you never mention the ones that didn’t. Do a little fraud // get a paper published // get down tonight.

同行评议的发明甚至可能鼓励了糟糕的研究。如果你试图发表一篇论文，表明看小狗视频会让人们向慈善机构捐赠更多，而审稿人2说:“如果这个结果也适用于猫，我才会被打动”，那么你就面临着极大的压力去做猫的视频研究。也许你会捏造一些数字，或者丢弃一些异常值。或者测试一堆猫视频，直到你找到一个有效的，然后你从不提及那些无效的。做点小骗局//发表一篇论文//今晚就完成。

PEER REVIEW, WE HARDLY TOOK YE SERIOUSLY

同行评议，我们几乎不把你当回事

Here’s another way that we can test whether peer review worked: did it actually earn scientists' trust?

还有另一种方法可以测试同行评议是否有效:它是否真的赢得了科学家的信任?

Scientists often say they take peer review very seriously. But people say lots of things they don’t mean, like “It’s great to e-meet you” and “I’ll never leave you, Adam.” If you look at what scientists actually do, it’s clear they don’t think peer review really matters.

科学家们经常说，他们非常重视同行评议。但人们会说很多言不由衷的话，比如"很高兴在网上认识你"和"我永远不会离开你，亚当"如果你看看科学家们实际上是怎么做的，很明显他们并不认为同行评议真的重要。

First: if scientists cared a lot about peer review, when their papers got reviewed and rejected, they would listen to the feedback, do more experiments, rewrite the paper, etc. Instead, they usually just submit the same paper to another journal. This was one of the first things I learned as a young psychologist, when my undergrad advisor explained there is a “big stochastic element” in publishing (translation: “it’s random, dude”). If the first journal didn’t work out, we’d try the next one. Publishing is like winning the lottery, she told me, and the way to win is to keep stuffing the box with tickets. When very serious and successful scientists proclaim that your supposed system of scientific fact-checking is no better than chance, that’s pretty dismal.

第一:如果科学家很在意同行评议，当他们的论文被评议和拒绝时，他们会听取反馈，做更多的实验，重写论文，等等。相反，他们通常只是将同一篇论文提交给另一家期刊。这是我作为一名年轻的心理学家学到的第一件事，当时我的本科导师解释说，出版中存在“很大的随机因素”。如果第一个期刊不成功，我们会尝试下一个。发表论文就像中彩票，她告诉我，要想中奖，就要不断地往盒子里塞彩票。当非常严肃和成功的科学家告诉你科学系统的真实—审查系统并不比随机瞎蒙好不了多少时，心情是相当令人沮丧的。

Second: once a paper gets published, we shred the reviews. A few journals publish reviews; most don't. Nobody cares to find out what the reviewers said or how the authors edited their paper in response, which suggests that nobody thinks the reviews actually mattered in the first place.

第二:一旦论文发表，评审意见就没用了。一些期刊发表评论，多数不。没有人关心审稿人说了什么，或者作者怎样应付审稿意见修改他们的文章已经无关重要，这表明没有人认为审稿意见有什么重要性。

And third: scientists take unreviewed work seriously without thinking twice. We read “preprints” and working papers and blog posts, none of which have been published in peer-reviewed journals. We use data from Pew and Gallup and the government, also unreviewed. We go to conferences where people give talks about unvetted projects, and we do not turn to each other and say, “So interesting! I can’t wait for it to be peer reviewed so I can find out if it’s true.”

第三:科学家们不假思索地认真对待未经审查的文章。我们阅读“预印本”、工作报告和博客文章，这些都不是发表在同行评议期刊上的东西。我们使用来自Pew、Gallup和政府的数据，同样未经审查。我们去参加会议，人们谈论那些尚未发表的工作，不会有人说，“这个想法太好!我迫不及待地想要等同行的评议的结果，以便决定这是不是一个科学真理。”

Instead, scientists tacitly agree that peer review adds nothing, and they make up their minds about scientific work by looking at the methods and results. Sometimes people say the quiet part loud, like Nobel laureate Sydney Brenner:

相反，科学家们心照不宣地同意同行评议没有任何好处，他们通过观察方法和结果来判断科学工作。有时人们会大声说出大家不说的东西，就像诺贝尔奖得主Sydney Brenner所述:

I don’t believe in peer review because I think it’s very distorted and as I’ve said, it’s simply a regression to the mean. I think peer review is hindering science. In fact, I think it has become a completely corrupt system.

我不相信同行评议，因为我认为它是非常扭曲的。正如我所说的，它只是回归到平庸。我认为同行评议阻碍了科学的发展。事实上，我认为它已经成为一个完全腐败的系统。

CAN WE FIX IT? NO WE CAN'T

同行评议能够改善吗？不能！

I used to think about all the ways we could improve peer review. Reviewers should look at the data! Journals should make sure that papers aren’t fraudulent!

我曾经想过改善同行评议的方法。审稿人应该看数据!期刊应该确保论文不是涉及伪造!

It’s easy to imagine how things could be better—my friend Ethan and I wrote a whole paper on it—but that doesn’t mean it’s easy to make things better. My complaints about peer review were a bit like looking at the ~35,000 Americans who die in car crashes every year and saying “people shouldn’t crash their cars so much.” Okay, but how?

想象事情如何变得更好是很容易的——我和朋友Ethan为此写了一篇论文——但这并不意味着很容易使事情变得更好。我对同行评议的悲观看法有点像看着每年死于车祸的3.5万美国人说:“人们不应该把车撞得这么惨。”好吧，但怎么做呢?

Lack of effort isn’t the problem: remember that our current system requires 15,000 years of labor every year, and it still does a really crappy job. Paying peer reviewers doesn’t seem to make them any better. Neither does training them. Maybe we can fix some things on the margins, but remember that right now we’re publishing papers that use capital T’s instead of error bars, so we’ve got a long, long way to go.

同行评议的问题并不是缺乏努力:我们目前的同行评议每年需要15000年的劳动，但它仍然很糟糕。付钱给同行审稿人似乎并没有让这些审稿人变得更好。培训审稿人也无济于事。也许我们可以在边边角角做点修复，但请记住，现在发表论文是用大写的T代替误差条，所以我们还有很长的路要走。

What if we made peer review way stricter? That might sound great, but it would make lots of other problems with peer review way worse.

如果我们让同行评议更严格呢?这听起来可能很棒，但它会使同行评议的许多其他问题变得更糟。

For example, you used to be able to write a scientific paper with style. Now, in order to please reviewers, you have to write it like a legal contract. Papers used to begin like, “Help! A mysterious number is persecuting me,” and now they begin like, “Humans have been said, at various times and places, to exist, and even to have several qualities, or dimensions, or things that are true about them, but of course this needs further study (Smergdorf & Blugensnout, 1978; Stikkiwikket, 2002; von Fraud et al., 2018b)”.

例如，你过去可以写一篇有风格的科学论文。现在，为了取悦审稿人，你必须把它写得像一份法律合同。以前报纸的开头是这样的:“救命!一个神秘的数字在折磨着我”, 现在他们开始这样说，“在不同的时间和地点，人们说人类存在，甚至有一些品质，或维度，或关于他们的真实的事情，但当然这需要进一步研究(Smergdorf & Blugensnout,1978;Stikkiwikket, 2002;von Fraud等人，2018b)

This blows. And as a result, nobody actually reads these papers. Some of them are like 100 pages long with another 200 pages of supplemental information, and all of it is written like it hates you and wants you to stop reading immediately. Recently, a friend asked me when I last read a paper from beginning to end; I couldn’t remember, and neither could he. “Whenever someone tells me they loved my paper,” he said, “I say thank you, even though I know they didn’t read it.” Stricter peer review would mean even more boring papers, which means even fewer people would read them.

很少打击，是不是。结果是，没有人真正去读这些论文。有些长达100页，还有200页的补充信息，所有这些都写的内容很讨厌你，在劝你放弃而不是鼓励你读下去。最近，一个朋友问我上一次从头到尾读完一篇论文是什么时候;我不记得了，他也不记得了。“每当有人对我说他们喜欢我的论文时”他说，“我都会说谢谢，即使我知道他们没有读我的论文。”更严格的同行评议意味着更多无聊的论文，这意味着更少的人会阅读它们

Making peer review harsher would also exacerbate the worst problem of all: just knowing that your ideas won’t count for anything unless peer reviewers like them makes you worse at thinking. It’s like being a teenager again: before you do anything, you ask yourself, “BUT WILL PEOPLE THINK I’M COOL?” When getting and keeping a job depends on producing popular ideas, you can get very good at thought-policing yourself into never entertaining anything weird or unpopular at all. That means we end up with fewer revolutionary ideas, and unless you think everything’s pretty much perfect right now, we need revolutionary ideas real bad.

让同行评议更严厉也会加剧最糟糕的问题:知道你的想法没有任何价值，只是审稿人喜欢你这么写。这使让你更不善于思考。这就像把你的智力拉回到十几岁的孩子:在你做任何事情之前，你都会问自己:“人们会认为我这样做很酷吗?”当获得并保住一份工作取决于产生流行的想法时，你可以很好地做到控制自己的思想，使自己从不享受任何奇怪或不同于流行的事情。这意味着最终会有更少的革命性想法。除非现在一切都很完美，否则我们真的非常需要革命性的想法。

On the off chance you do figure out a way to improve peer review without also making it worse, you can try convincing the nearly 30,000 scientific journals in existence to apply your magical method to the ~4.7 million articles they publish every year. Good luck!

如果你真的有运气想出一种既能改善同行评议又不会让它变得更糟的方法，你可以试着说服现有的近3万家科学期刊，把你的神奇方法应用到它们每年发表的约470万篇文章上。祝你好运！

PEER REVIEW IS WORSE THAN NOTHING; OR, WHY IT AIN’T ENOUGH TO SNIFF THE BEEF

同行评议比没有同行评议更糟糕;换言之，为什么闻一下牛肉是远远不够的。

Peer review doesn’t work and there’s probably no way to fix it. But a little bit of vetting is better than none at all, right?

同行评议没有用，而且可能没有办法改善同行评议。但有一点审查总比没有好，对吗?

I say: no way.

不对！

Imagine you discover that the Food and Drug Administration’s method of “inspecting” beef is just sending some guy (“Gary”) around to sniff the beef and say whether it smells okay or not, and the beef that passes the sniff test gets a sticker that says “INSPECTED BY THE FDA.” You’d be pretty angry. Yes, Gary may find a few batches of bad beef, but obviously he’s going to miss most of the dangerous meat. This extremely bad system is worse than nothing because it fools people into thinking they’re safe when they’re not.

设想一下，你发现食品和药物管理局(Food and Drug Administration)“检查”牛肉的方法只是派一个人(“Gary”)去闻牛肉，看看它是否闻起来还行。通过嗅测的牛肉会得到一个标签，上面写着“经FDA检查”。 你会作何感想？是的，Gary可能会发现几批坏牛肉，但显然他会错过大多数危险的肉。这个极其糟糕的系统比什么都没有还糟糕，因为它让人们误以为自己是安全的，而实际上他们并不安全。

That’s what our current system of peer review does, and it’s dangerous. That debunked theory about vaccines causing autism comes from a peer-reviewed paper in one of the most prestigious journals in the world, and it stayed there for twelve years before it was retracted. How many kids haven’t gotten their shots because one rotten paper made it through peer review and got stamped with the scientific seal of approval?

这就是目前的同行评议系统所做的，这是危险的。这个被揭穿的关于疫苗导致自闭症的理论来自一篇同行评议的论文，发表在世界上最负盛名的期刊之一上。12年之后才被撤回。有多少孩子因为一篇烂论文通过了同行评议并盖上了科学认可章而没有接种疫苗?

If you want to sell a bottle of vitamin C pills in America, you have to include a disclaimer that says none of the claims on the bottle have been evaluated by the Food and Drug Administration. Maybe journals should stamp a similar statement on every paper: “NOBODY HAS REALLY CHECKED WHETHER THIS PAPER IS TRUE OR NOT. IT MIGHT BE MADE UP, FOR ALL WE KNOW.” That would at least give people the appropriate level of confidence.

如果你想在美国销售一瓶维生素C药片，你必须附上一份免责声明，说明瓶子上的任何声明都没有经过美国食品和药物管理局的评估。也许期刊应该在每篇论文上都贴上类似的声明:“没有人真正检查过这篇论文是不是对。据我们所知，这可能是编造的。” 这至少会让人们保持适度的警惕。

SCIENCE MUST BE FREE

科学必须是自由的

Why did peer review seem so reasonable in the first place?

为什么同行评议从一开始就显得如此合理?

I think we had the wrong model of how science works. We treated science like it’s a weak-link problem where progress depends on the quality of our worst work. If you believe in weak-link science, you think it’s very important to stamp out untrue ideas—ideally, prevent them from being published in the first place. You don’t mind if you whack a few good ideas in the process, because it’s so important to bury the bad stuff.

我们对科学进步的模型是错误的。我们以为科学进步遵循短板理论，即改善最糟糕的工作才能提升整体质量。如果你相信短板理论，你会认为排除不良的研究对期刊论文非常重要——最好从一开始就阻止它们被发表。你不介意在这个过程中同时挖掉了一些好的思想，因为把不好的东西埋掉很重要的。

But science is a strong-link problem: progress depends on the quality of our best work. Better ideas don’t always triumph immediately, but they do triumph eventually, because they’re more useful. You can’t land on the moon using Aristotle’s physics, you can’t turn mud into frogs using spontaneous generation, and you can’t build bombs out of phlogiston. Newton’s laws of physics stuck around; his recipe for the Philosopher’s Stone didn’t. We didn’t need a scientific establishment to smother the wrong ideas. We needed it to let new ideas challenge old ones, and time did the rest.

但科学进步符合长板理论: 科学进步取决于我们最好工作的质量。更好的想法不一定会马上胜利，但它们最终会胜利，因为它们更有用。你不能用亚里士多德的物理学登上月球，你不能用自发的化学反应把泥变成青蛙，你不能用燃素制造炸弹。牛顿的物理定律可能被阻碍;但是他的哲学不能被阻碍的柱。我们不需要一个科学机构来扼杀错误的想法。我们需要让新思想挑战旧思想，剩下的就交给时间了。

If you’ve got weak-link worries, I totally get it. If we let people say whatever they want, they will sometimes say untrue things, and that sounds scary. But we don’t actually prevent people from saying untrue things right now; we just pretend to. In fact, right now we occasionally bless untrue things with big stickers that say “INSPECTED BY A FANCY JOURNAL,” and those stickers are very hard to get off. That’s way scarier.

如果你担心的是薄弱环节，那并没有错。如果我们让人们想说什么就说什么，他们有时会说慌，这听起来很可怕。但同行评审并没有阻止人们说慌;我们只是假装做到了没有让人说谎。事实上，我们经常会用大贴纸为谎言贴金，大贴纸上面写着“被一本高档的杂志审查过”，而这些贴纸很难被撕下来。那样才是更可怕的。

Weak-link thinking makes scientific censorship seem reasonable, but all censorship does is make old ideas harder to defeat. Remember that it used to be obviously true that the Earth is the center of the universe, and if scientific journals had existed in Copernicus’ time, geocentrist reviewers would have rejected his paper and patted themselves on the back for preventing the spread of misinformation. Eugenics used to be hot stuff in science—do you think a bunch of racists would give the green light to a paper showing that Black people are just as smart as white people? Or any paper at all by a Black author? (And if you think that’s ancient history: this dynamic is still playing out today.) We still don’t understand basic truths about the universe, and many ideas we believe today will one day be debunked. Peer review, like every form of censorship, merely slows down truth.

短板理论使科学审查看起来合理，但审查只是使旧思想更难被推翻。请记住，地球是宇宙的中心曾经是显而易见的事实。如果同行评审存在于哥白尼的时代，地球中心论的审稿人会拒绝他的论文，并且因为防止了“错误信息”的传播而拍拍自己的后背。优生学曾经是科学界的热门话题——你认为一群种族主义者会对一篇表明黑人和白人一样聪明的论文或者任何一篇黑人作者的论文开绿灯吗?(如果你认为这是古老的历史，但是它今天仍在上演。)我们仍然不了解宇宙的基本真相，我们今天相信的许多错误观点总有一天会被证伪。同行评议，与任何形式的审查一样，只会减慢得到真理的速度。

HOORAY WE FAILED

万岁，同行评审的实验失败了

Nobody was in charge of our peer review experiment, which means nobody has the responsibility of saying when it’s over. Seeing no one else, I guess I’ll do it:

没有人为同行评议实验负责，这意味着没有人告诉你什么时候这场实验结束。既然没有人宣布这场实验结束，我不介意来做这件事。

We’re done, everybody! Champagne all around! Great work, and congratulations. We tried peer review and it didn’t work.

事情结束了！香槟!干得好! 恭喜! 我们试过同行评议，但是没有成功。

Honesty, I’m so relieved. That system sucked! Waiting months just to hear that an editor didn’t think your paper deserved to be reviewed? Reading long walls of text from reviewers who for some reason thought your paper was the source of all evil in the universe? Spending a whole day emailing a journal begging them to let you use the word “years” instead of always abbreviating it to “y” for no reason (this literally happened to me)? We never have to do any of that ever again.

说实话，我松了一口气。同行评审糟透了!等了几个月，就为了听到一个编辑认为你的论文不值得评审?阅读那些出于某种原因认为你的论文是宇宙万恶之源的评论家们的长篇大论?花一整天的时间给杂志发邮件，请求他们允许你使用“years”这个词，而不是总是毫无理由地把它缩写为“y”(这真的发生在我身上)?我们再也不用这么做了。

I know we all might be a little disappointed we wasted so much time, but there's no shame in a failed experiment. Yes, we should have taken peer review for a test run before we made it universal. But that’s okay—it seemed like a good idea at the time, and now we know it wasn’t. That’s science! It will always be important for scientists to comment on each other’s ideas, of course. It’s just this particular way of doing it that didn’t work.

我知道我们都有点失望我们浪费了这么多时间，但实验失败也没什么丢人的。是的，在普及同行评议之前，我们本来应该先对同行评议实验预先进行一次同行评议。但没关系——当时同行评议似乎是个好主意，现在我们知道它不是。这就是科学!当然，对科学家来说，评论彼此的观点总是很重要的。只是这种同行评议的方法行不通。

What should we do now? Well, last month I published a paper, by which I mean I uploaded a PDF to the internet. I wrote it in normal language so anyone could understand it. I held nothing back—I even admitted that I forgot why I ran one of the studies. I put jokes in it because nobody could tell me not to. I uploaded all the materials, data, and code where everybody could see them. I figured I’d look like a total dummy and nobody would pay any attention, but at least I was having fun and doing what I thought was right.

我们现在该怎么办?嗯，上个月我发表了一篇论文，我的意思是我把PDF上传到网上。我用普通语言写的，这样任何人都能看懂。我没有任何保留——我甚至承认我忘记了我为什么要做其中一项研究。我在里面放了笑话，因为没人能告诉我不能这样做。我把所有材料，数据和代码都上传到了大家都能看到的地方。我想我会看起来像个傻瓜，没有人会注意到，但至少我很开心，做着我认为正确的事情。

Then, before I even told anyone about the paper, thousands of people found it, commented on it, and retweeted it.

然后，在我告诉任何人这篇论文之前，成千上万的人发现了它，评论了它，并反复推特了它。

This is how fun and genuine science could be! Can't believe there is a paper that I could finish reading at 6:30 am laying in bed - if that helps prove how stunning this work is. Absolutely recommend!

这才是真正有趣的科学!真不敢相信我能在早上6点半躺在床上读完一篇论文——如果这有助于证明这项工作有多惊人的话。绝对推荐!

Things could be better

事情本可以变得更好

Likes

喜欢

Ugh why don't we write papers this way?! I'd write up all my studies so fast if I could just write in layperson honesty. So easy to understand and actually fun to read!

我们为什么不这样写论文呢?如果我能以外行的诚实写作，我就会很快写完我所有的研究报告。如此容易理解，实际上有趣的阅读!

This may be the best paper I’ve ever read

这可能是我读过的最好的论文

Total strangers emailed me thoughtful reviews. Tenured professors sent me ideas. NPR asked for an interview. The paper now has more views than the last peer-reviewed paper I published, which was in the prestigious Proceedings of the National Academy of Sciences. And I have a hunch far more people read this new paper all the way to the end, because the final few paragraphs got a lot of comments in particular. So I dunno, I guess that seems like a good way of doing it?

完全陌生的人给我发了有见解的评论邮件。终身教授给我发了一些想法。美国国家公共电台要求采访。这篇论文现在比我上一篇发表在著名的《美国国家科学院院刊》(Proceedings of The National Academy of Sciences，PNAS)上的同行评审论文拥有更多的浏览量。我有一种直觉，更多的人会从头读到尾，因为最后几段得到了很多特别的评论。我想这似乎是一个很好的发表观点的方法?

I don’t know what the future of science looks like. Maybe we’ll make interactive papers in the metaverse or we’ll download datasets into our heads or whisper our findings to each other on the dance floor of techno-raves. Whatever it is, it’ll be a lot better than what we’ve been doing for the past sixty years. And to get there, all we have to do is what we do best: experiment.

我不知道科学的未来会是什么样子。也许我们会在虚拟世界里制作互动式论文，或者我们会把数据集下载到我们的大脑里，或者在科技狂欢的舞池里互相窃窃私语我们的发现。不管它是什么，它将比我们过去60年来所做的要好得多。为了达到这个目标，我们所要做的就是我们最擅长的:实验。

I should also note that this all fits with Max Weber's theory that specialist classes make informational gatekeeping processes into safeguards for their own status.

我还应该指出，这一切都符合马克斯•韦伯(Max Weber)的理论，即专业阶层将信息把关过程变成了保护他们自身地位的保障。