Reviewers’ scores do not predict impact: bibliometric analysis of the proceedings of the human–robot interaction conference

Christoph Bartneck
Received: 10 June 2016
Akade´miai Kiado´ , Budapest, Hungary 2016

Abstract： The peer review process is an essential component for the progress of science.The ACM/IEEE International Conference on Human–Robot Interaction is the prime publication channel for the field and this study evaluates its peer review process. The results show that the number of accepted papers are unevenly distributed across countries,organizations and authors. The contributions from the US outweigh all others contributions.A Binary Logistic Regression analysis showed that only for 85.5% of the papers the reviewers’ scores accurately predict its acceptance or rejection. Moreover, there was no significant correlation between the reviewers’ scores and the citations the papers later attract. 73% of the authors only ever submitted one paper and the proportion of newcomers at the conferences ranges from 63–77%.

（同行评议过程是科学发展过程中一个非常重要的组成部分。人机交互ACM/IEEE国际会议是这个领域重要的发表渠道。这项研究是要评估同行评议过程。研究结果表明，会议接受的论文在国家、机构和作者层面都呈现不均衡的分布。美国作者的贡献远超其它国家作者的贡献。通过二元逻辑回归分析表明，85.5%的论文的评议能够准确地预测论文的接收与拒绝。而且，评议的分值和论文被引没有明显的相关性。73%的作者只提交过一篇论文，而且每次新参加会议的作者大概占总人数的63%到77%。 ）

Keywords Human–robot interaction  HRI  Conference  Proceedings  Acceptance rate

This study performs a bibliomentric analysis of the accepted and rejected papers of the HRI conference. More specifically, we are interesting in answering the following research questions:
1. How did the conference develop in terms of submissions and acceptance rate?（根据提交论文数和论文接收率看会议的发展状况？）
2. What is the acceptance rate depending on the number of authors on an paper?（论文接收率是否依赖于论文作者数量？）
3. Does collaboration across organization increase citations?（跨组织的合作是否会提升被引次数？）
4. Do papers from some countries get accepted more often than papers from other countries?（某些国家稿件接收是否多于其它国家？）
5. What are the most successful organizations?（最成功的组织是？）
6. What is the distribution of accepted and rejected papers across authors?（不同作者论文接收与拒稿的分布？）
7. What is the distribution of accepted and rejected papers across topics?（不同主题论文接收与拒稿的分布？）
8. What is the relationship between the reviewers’ scores and the acceptance of an paper?（同行评议分值跟论文接收之间关系？）
9. What is the relationship between the reviewers’ scores and the citations papers receive?（同行评议分值跟论文被引之间关系？）
10. What is the author retention of the conference?（会议作者的维持情况？）

Reviewers’ scores do not predict impact.pdf