Information geometry is an interdisciplinary field that applies the techniques of differential geometry to study probability theory and statistics. It studies statistical manifolds, which are Riemannian manifolds whose points correspond to probability distributions.
Introduction
Historically, information geometry can be traced back to the work of C. R. Rao, who was the first to treat the Fisher matrix as a Riemannian metric.[1][2] The modern theory is largely due to Shun'ichi Amari, whose work has been greatly influential on the development of the field.[citation needed]
Classically, information geometry considered a parametrized statistical model as a Riemannian manifold. For such models, there is a natural choice of Riemannian metric, known as the Fisher information metric. In the special case that the statistical model is an exponential family, it is possible to induce the statistical manifold with a Hessian metric (i.e a Riemannian metric given by the potential of a convex function). In this case, the manifold naturally inherits two flat affine connections, as well as a canonical Bregman divergence. Historically, much of the work was devoted to studying the associated geometry of these examples. In the modern setting, information geometry applies to a much wider context, including non-exponential families, nonparametric statistics, and even abstract statistical manifolds not induced from a known statistical model. The results combine techniques from information theory, affine differential geometry, convex analysis and many other fields.
The standard references in the field are Shun’ichi Amari and Hiroshi Nagaoka's book, Methods of Information Geometry,[3] and the more recent book by Nihat Ay and others.[4] A gentle introduction is given in the survey by Frank Nielsen.[5] In 2018, the journal Information Geometry was released, which is devoted to the field.
Contributors
The history of information geometry is associated with the discoveries of at least the following people, and many others.
Ronald Fisher
Harald Cramér
Calyampudi Radhakrishna Rao
Harold Jeffreys
Solomon Kullback
Jean-Louis Koszul
Richard Leibler
Claude Shannon
Imre Csiszár
N. N. Cencov (also written as Chentsov)
Bradley Efron
Shun'ichi Amari
Ole Barndorff-Nielsen
Frank Nielsen
Damiano Brigo
A. W. F. Edwards
Grant Hillier
Kees Jan Van Garderen
Applications
As an interdisciplinary field, information geometry has been used in various applications.
Here an incomplete list:
Statistical inference
Time series and linear systems
Quantum systems
Neural networks
Machine learning
Statistical mechanics
Biology
Statistics
Mathematical finance
See also
Ruppeiner geometry
Kullback–Leibler divergence
References
Rao, C. R. (1945). "Information and Accuracy Attainable in the Estimation of Statistical Parameters". Bulletin of the Calcutta Mathematical Society. 37: 81–91. Reprinted in Breakthroughs in Statistics. Springer. 1992. pp. 235–247. doi:10.1007/978-1-4612-0919-5_16.
Nielsen, F. (2013). "Cramér-Rao Lower Bound and Information Geometry". In Bhatia, R.; Rajan, C. S. (eds.). Connected at Infinity II: On the Work of Indian Mathematicians. Special Volume of Texts and Readings in Mathematics (TRIM). Hindustan Book Agency. arXiv:1301.3578. ISBN 978-93-80250-51-9.
Amari, Shun'ichi; Nagaoka, Hiroshi (2000). Methods of Information Geometry. Translations of Mathematical Monographs. 191. American Mathematical Society. ISBN 0-8218-0531-2.
Ay, Nihat; Jost, Jürgen; Lê, Hông Vân; Schwachhöfer, Lorenz (2017). Information Geometry. Ergebnisse der Mathematik und ihrer Grenzgebiete. 64. Springer. ISBN 978-3-319-56477-7.
Nielsen, Frank (2018). "An Elementary Introduction to Information Geometry". arXiv:1808.08271.
Further reading
Amari, Shun'ichi (1985). Differential-Geometrical Methods in Statistics. Lecture Notes in Statistics. Berlin: Springer-Verlag. ISBN 0-387-96056-2.
Murray, M.; Rice, J. (1993). Differential Geometry and Statistics. Monographs on Statistics and Applied Probability. 48. Chapman and Hall. ISBN 0-412-39860-5.
Kass, R. E.; Vos, P. W. (1997). Geometrical Foundations of Asymptotic Inference. Series in Probability and Statistics. Wiley. ISBN 0-471-82668-5.
Marriott, Paul; Salmon, Mark, eds. (2000). Applications of Differential Geometry to Econometrics. Cambridge University Press. ISBN 0-521-65116-6.
Undergraduate Texts in Mathematics
Graduate Studies in Mathematics
Hellenica World - Scientific Library
Retrieved from "http://en.wikipedia.org/"
All text is available under the terms of the GNU Free Documentation License
