博文
Deviation of Zipf’s and Heaps’ Laws in Human Languages
|||
Deviation of Zipf’s and Heaps’ Laws in Human Languages with Limited Dictionary Sizes
Zipf’s law on word frequency and Heaps’ law on the growth of distinct words are observed in Indo-European
language family, but it does not hold for languages like Chinese, Japanese and Korean. These languages
consist of characters, and are of very limited dictionary sizes. Extensive experiments show that: (i) The
character frequency distribution follows a power law with exponent close to one, at which the corresponding
Zipf’s exponent diverges. Indeed, the character frequency decays exponentially in the Zipf’s plot. (ii) The
number of distinct characters grows with the text length in three stages: It grows linearly in the beginning,
then turns to a logarithmical form, and eventually saturates. A theoretical model for writing process is
proposed, which embodies the rich-get-richer mechanism and the effects of limited dictionary size.
Experiments, simulations and analytical solutions agree well with each other. This work refines the
understanding about Zipf’s and Heaps’ laws in human language systems.
Citation：Linyuan Lu，Zi-Ke Zhang， Tao Zhou，Deviation of Zipf’s and Heaps’ Laws in Human Languages with Limited Dictionary Sizes，Scientific Reports 3，1082 （2013）.
Download:srep01082.pdf
最近有一文，较详细讨论了Heaps和Zipf之关系，感兴趣者可供参考~
题目：Power-Law Connections: From Zipf to Heaps and Beyond
作者：Iddo I. Eliazar and Morrel H. Cohen
Abstract
In this paper we explore the asymptotic statistics of a general model of rank distribution in the large-ensemble limit; the construction of the general model is motivated by recent empirical studies of rank distributions. Applying Lorenzian, oligarchic, and Heapsian asymptotic analyses we establish a comprehensive set of closed-form results linking together rank distributions, probability distributions, oligarchy sizes, and innovation rates. In particular, the general results reveal the fundamental underlying connections between Zipf’s law, Pareto’s law, and Heaps’ law– three elemental empirical power-laws that are ubiquitously observed in the sciences.
Keywords: rank distributions; power-laws; Zipf’s law; Pareto’s law; Heaps’ law; Lorenz curves; the distribution of wealth; oligarchy sizes; innovation rates; phase transitions; self-organized criticality.
PACS: 02.50.-r (Probability theory, stochastic processes, and statistics);
89.65.-s (Social and economic systems).
https://blog.sciencenet.cn/blog-329471-664970.html
上一篇：Potential Theory for Directed Networks
下一篇：第14届特伦托暑期班-Modularity and Design for Innovation
