Innovation indices: the need for positioning them where they properly belong

创新指标需要各得其所

Scientometrics（DOI 10.1007/s11192-015-1632-4）

Jan Kozłowski在对相关研究的基础上，从反实证主义趋势、过量的指标、创新概念的争论、指标的争论和解决办法5个方面对创新指标的相关问题进行了论述。从作者的调研看，过去10年已经有150多个创新指标，p 些指标引起了不少争论。我觉得本文对创新指标的相关研究进行了不同角度的归纳，其中的一些理解和认识对于各类评价研究有一定的指导意义。

以下为内容摘自论文

For the purpose of our paper, these propositions may be divided into three (interrelated)

groups:

• propositions about the lack of neutrality in the language of inquiry,（调查缺乏中间性）

• propositions about the possibility of achieving ‘the one, definitive’ perspective,（达成明确意向的可能性）

• propositions confirming that the cognitive content is a ‘self-fulfilling prophecy（认知内容是一个自我实现预言）

Over the years, indicators were differentiated in terms of forms (structural, input,process, output, outcome, impact) and applications (budget; control; evaluation; implementation;management; milestone; monitoring; policy; planning; programme; project;resource efficiency; roadmap; strategy…).

各种指标以不同形式进行区分（结构化的，输入、处理、输出、产出，影响）和应用（预算、控制、评价、完善、管理、里程碑、政策、计划、程序、目标、资源有效性、路径、战略等）

If indicators represent a second-order abstraction and packaging of raw statistical data (Davis et al. 2010), the group of composite indicators, indices, scoreboards and rankings represent third-order.（如果原始统计数据是第一层次；指标是对原始统计数据的抽象和包装一的第二层次；那么，复合指标、指标体系、记分板、排序是第三个层次）The last 20 years have seen growing interest in the development and use of these third order

tools to improve the way innovation system features are measured and described.（近20年来，研究者比较关注用第三个层次来改善创新系统）

If we are so critical of indicators, what should we do about it or what replace them with?

There are two solutions, one (moderate) in the short run, the second (radical) in the long

run.（对于创新指标有很多争论，我们如何来做？可以短期和长期两种解决办法）。

As concerns the short-run solution, three options are discussed.

First, to leave the situation as it is, but to strengthen the search for better indicators and

better indices.（加强或寻找更好的指标）

Second, abolishing indices and thus generating pressure to work on new econometric

studies and policy studies, funded as a separate EU program.（放弃指标体系而指它转移给新的经济学和政策研究，如独立的EU项目。）

Third, withdrawing the indices from centre stage in order to deprive policies of their real

target and shifting them to the measurable components of the indices.（？）

A radical long-term solution is associated with Big Data. A tool that will replace

innovation indices will be based on big data. Big data have changed the way that we live,

do business, manage organizations and carry out research. Big data are increasingly the

fuel for and the driver of economic growth and form the basis for new research.

（长期的举措和大数据有关。用基于大数据的工具来代替创新指数。大数据正在改变着我们的生活、商务活动、组织管理和研究。大数据是驱动经济增长和形成新的研究的基础。）

（文章还没有安全看懂，以上的有些翻译并不准确，仅供参考。）

Abstract：A specific quality of the discussion about innovation indices (scoreboards) is that more often than not the subject is dealt with from a purely technical point of view. Such a narrow approach silently assumes that indices used as a policy tool are an accurate reflection of the phenomenon and should not be questioned, and also that the whole discussion concerning them should refer to methodological aspects and is best left to the statisticians. This author is of the opinion that for an accurate evaluation of the value of indices as a policy tool, it is necessary to consider the matter from the broader point of view and from the context in which such indices are generated and used. This article puts forward the thesis that progress in science and innovation policy studies depends on a diversity of issues, approaches and perspectives. If that is the case, maintaining thematic and methodological variety may be more important than creating coherent and closed analytical tools, i.e. indices. The advantage of indices is that they focus attention on those variables which are deemed to be key. Among their disadvantages, however, are their highly abstract nature (in order to understand innovation-related phenomena, it is necessary to study them in tangible, composite forms); their tendency to skip unmeasurable determinants; their prior acceptance of definitions and concepts of innovation (instead of searching for them); the way they apply a single yardstick to diverse countries and regions, assumed linearity and causality in a complex and non-linear world, the way they direct policy towards implementing indicators (rather than identifying and solving problems). It is suggested that big data revolution will allow the emergence of a new measurement tools that will replace innovation indices.

Innovation indices.pdf