In probability and statistics, the Yule–Simon distribution is a discrete probability distribution named after Udny Yule and Herbert A. Simon. Simon originally called it the Yule distribution.[1]
The probability mass function (pmf) of the Yule–Simon (ρ) distribution is
\( {\displaystyle f(k;\rho )=\rho \operatorname {B} (k,\rho +1),} \)
for integer \( k\geq 1 \) and real \( \rho >0 \), where \( {\displaystyle \operatorname {B} } \) is the beta function. Equivalently the pmf can be written in terms of the rising factorial as
\( {\displaystyle f(k;\rho )={\frac {\rho \Gamma (\rho +1)}{(k+\rho )^{\underline {\rho +1}}}},} \)
where \( \Gamma \) is the gamma function. Thus, if \( \rho \) is an integer,
\( {\displaystyle f(k;\rho )={\frac {\rho \,\rho !\,(k-1)!}{(k+\rho )!}}.} \)
The parameter ρ {\displaystyle \rho } \rho can be estimated using a fixed point algorithm.[2]
The probability mass function f has the property that for sufficiently large k we have
\( {\displaystyle f(k;\rho )\approx {\frac {\rho \Gamma (\rho +1)}{k^{\rho +1}}}\propto {\frac {1}{k^{\rho +1}}}.} \)
Plot of the Yule–Simon(1) distribution (red) and its asymptotic Zipf's law (blue)
This means that the tail of the Yule–Simon distribution is a realization of Zipf's law: \( f(k;\rho ) \) can be used to model, for example, the relative frequency of the kth most frequent word in a large collection of text, which according to Zipf's law is inversely proportional to a (typically small) power of k.
Occurrence
The Yule–Simon distribution arose originally as the limiting distribution of a particular stochastic process studied by Yule as a model for the distribution of biological taxa and subtaxa.[3] Simon dubbed this process the "Yule process" but it is more commonly known today as a preferential attachment process. The preferential attachment process is an urn process in which balls are added to a growing number of urns, each ball being allocated to an urn with probability linear in the number the urn already contains.
The distribution also arises as a compound distribution, in which the parameter of a geometric distribution is treated as a function of random variable having an exponential distribution. Specifically, assume that W {\displaystyle W} W follows an exponential distribution with scale \( 1/\rho \) or rate \( \rho \) :
\( {\displaystyle W\sim \operatorname {Exponential} (\rho ),} \)
with density
\( {\displaystyle h(w;\rho )=\rho \exp(-\rho w).}
Then a Yule–Simon distributed variable K has the following geometric distribution conditional on W:
\( {\displaystyle K\sim \operatorname {Geometric} (1-\exp(-W))\,.} \)
The pmf of a geometric distribution is
\( {\displaystyle g(k;p)=p(1-p)^{k-1}} \)
for \( {\displaystyle k\in \{1,2,\dotsc \}} \). The Yule–Simon pmf is then the following exponential-geometric compound distribution:
\({\displaystyle f(k;\rho )=\int _{0}^{\infty }g(k;\exp(-w))h(w;\rho )\,dw.} \)
The maximum likelihood estimator for the parameter \( \rho \) given the observations \( {\displaystyle k_{1},k_{2},k_{3},\dots ,k_{N}} \) is the solution to the fixed point equation
\( {\displaystyle \rho ^{(t+1)}={\frac {N+a-1}{b+\sum _{i=1}^{N}\sum _{j=1}^{k_{i}}{\frac {1}{\rho ^{(t)}+j}}}},} \)
where \( {\displaystyle b=0,a=1} \)are the rate and shape parameters of the gamma distribution prior on \( \rho \) .
This algorithm is derived by Garcia [2] by directly optimizing the likelihood. Roberts and Roberts [4]
generalize the algorithm to Bayesian settings with the compound geometric formulation described above. Additionally, Roberts and Roberts [4] are able to use the Expectation Maximisation (EM) framework to show convergence of the fixed point algorithm. Moreover, Roberts and Roberts [4] derive the sub-linearity of the convergence rate for the fixed point algorithm. Additionally, they use the EM formulation to give 2 alternate derivations of the standard error of the estimator from the fixed point equation. The variance of the \( \lambda \) estimator is
\( {\displaystyle \operatorname {Var} ({\hat {\lambda }})={\frac {1}{{\frac {N}{{\hat {\lambda }}^{2}}}-\sum _{i=1}^{N}\sum _{j=1}^{k_{i}}{\frac {1}{({\hat {\lambda }}+j)^{2}}}}},} \)
the standard error is the square root of the quantity of this estimate divided by N.
Generalizations
The two-parameter generalization of the original Yule distribution replaces the beta function with an incomplete beta function. The probability mass function of the generalized Yule–Simon(ρ, α) distribution is defined as
\( {\displaystyle f(k;\rho ,\alpha )={\frac {\rho }{1-\alpha ^{\rho }}}\;\mathrm {B} _{1-\alpha }(k,\rho +1),\,} \)
with \( 0\leq \alpha <1. \) For \( \alpha =0 \) the ordinary Yule–Simon(ρ) distribution is obtained as a special case. The use of the incomplete beta function has the effect of introducing an exponential cutoff in the upper tail.
See also
Zeta distribution
Scale-free network
Bibliography
Colin Rose and Murray D. Smith, Mathematical Statistics with Mathematica. New York: Springer, 2002, ISBN 0-387-95234-9. (See page 107, where it is called the "Yule distribution".)
References
Simon, H. A. (1955). "On a class of skew distribution functions". Biometrika. 42 (3–4): 425–440. doi:10.1093/biomet/42.3-4.425.
Garcia Garcia, Juan Manuel (2011). "A fixed-point algorithm to estimate the Yule-Simon distribution parameter". Applied Mathematics and Computation. 217 (21): 8560–8566. doi:10.1016/j.amc.2011.03.092.
Yule, G. U. (1924). "A Mathematical Theory of Evolution, based on the Conclusions of Dr. J. C. Willis, F.R.S". Philosophical Transactions of the Royal Society B. 213 (402–410): 21–87. doi:10.1098/rstb.1925.0002.
Roberts, Lucas; Roberts, Denisa (2017). "An Expectation Maximization Framework for Preferential Attachment Models". arXiv:1710.08511 [stat.CO].
Undergraduate Texts in Mathematics
Graduate Studies in Mathematics
Hellenica World - Scientific Library
Retrieved from "http://en.wikipedia.org/"
All text is available under the terms of the GNU Free Documentation License
