PyMC3

PyMC3 is a Python package for Bayesian statistical modeling and probabilistic machine learning which focuses on advanced Markov chain Monte Carlo and variational fitting algorithms.[1][2][3] It is a rewrite from scratch of the previous version of the PyMC software.[4] Unlike PyMC2, which had used Fortran extensions for performing computations, PyMC3 relies on Theano for automatic differentiation and also for computation optimization and dynamic C compilation.[2][5]. From version 3.8 PyMC3 relies on ArviZ to handle plotting, diagnostics, and statistical checks. PyMC3 and Stan are the two most popular probabilistic programming tools.[6] PyMC3 is an open source project, developed by the community and fiscally sponsored by NumFocus.[7]

PyMC3 has been used to solve inference problems in several scientific domains, including astronomy,[8][9] molecular biology,[10] crystallography,[11][12] chemistry,[13] ecology[14][15] and psychology.[16] Previous versions of PyMC were also used widely, for example in climate science,[17] public health,[18] neuroscience,[19] and parasitology.[20][21]

After Theano announced plans to discontinue development in 2017,[22] the PyMC3 team decided in 2018 to develop a new version of PyMC named PyMC4, and pivot to TensorFlow Probability as its computational backend. Until the new version is in beta, PyMC3 will continue to be the primary target of development efforts, and both it, and Theano as its backend, will be supported by the PyMC3 team for an extended period of time.[23][24]

Inference engines

PyMC3 implements non-gradient-based and gradient-based Markov chain Monte Carlo (MCMC) algorithms for Bayesian inference and stochastic, gradient-based variational Bayesian methods for approximate Bayesian inference.

MCMC algorithms:
No-U-Turn sampler[25] (NUTS), a variant of Hamiltonian Monte Carlo and PyMC3's default engine for continuous variables
Metropolis–Hastings, PyMC3's default engine for discrete variables
Sequential Monte Carlo
Variational inference algorithms:
Black-box Variational Inference[26]

See also

Stan is a probabilistic programming language for statistical inference written in C++

References

Salvatier J, Wiecki TV, Fonnesbeck C. (2016) Probabilistic programming in Python using PyMC3. PeerJ Computer Science 2:e55 https://doi.org/10.7717/peerj-cs.55
Martin, Osvaldo (2016). Bayesian Analysis with Python. Packt Publishing Ltd. pp. 31–60. ISBN 9781785889851. Retrieved 16 September 2017.
Davidson-Pilon, Cameron (2015-09-30). Bayesian Methods for Hackers: Probabilistic Programming and Bayesian Inference. Addison-Wesley Professional. ISBN 9780133902921.
"Introduction — PyMC3 3.1 documentation". docs.pymc.io. Retrieved 2017-09-20.
Hilpisch, Yves (2014-12-11). Python for Finance: Analyze Big Financial Data. O'Reilly Media, Inc. ISBN 9781491945391.
"The Algorithms Behind Probabilistic Programming". Retrieved 2017-03-10.
"NumFOCUS Announces New Fiscally Sponsored Project: PyMC3". NumFOCUS | Open Code = Better Science. Retrieved 2017-03-10.
Greiner, J.; Burgess, J. M.; Savchenko, V.; Yu, H.-F. (2016). "On the Fermi-GBM Event 0.4 s after GW150914". The Astrophysical Journal Letters. 827 (2): L38. arXiv:1606.00314. Bibcode:2016ApJ...827L..38G. doi:10.3847/2041-8205/827/2/L38. ISSN 2041-8205.
Hilbe, Joseph M.; Souza, Rafael S. de; Ishida, Emille E. O. (2017-04-30). Bayesian Models for Astrophysical Data: Using R, JAGS, Python, and Stan. Cambridge University Press. ISBN 9781108210744.
Wagner, Stacey D.; Struck, Adam J.; Gupta, Riti; Farnsworth, Dylan R.; Mahady, Amy E.; Eichinger, Katy; Thornton, Charles A.; Wang, Eric T.; Berglund, J. Andrew (2016-09-28). "Dose-Dependent Regulation of Alternative Splicing by MBNL Proteins Reveals Biomarkers for Myotonic Dystrophy". PLOS Genetics. 12 (9): e1006316. doi:10.1371/journal.pgen.1006316. ISSN 1553-7404. PMC 5082313. PMID 27681373.
Sharma, Amit; Johansson, Linda; Dunevall, Elin; Wahlgren, Weixiao Y.; Neutze, Richard; Katona, Gergely (2017-03-01). "Asymmetry in serial femtosecond crystallography data". Acta Crystallographica Section A. 73 (2): 93–101. doi:10.1107/s2053273316018696. ISSN 2053-2733. PMC 5332129. PMID 28248658.
Katona, Gergely; Garcia-Bonete, Maria-Jose; Lundholm, Ida (2016-05-01). "Estimating the difference between structure-factor amplitudes using multivariate Bayesian inference". Acta Crystallographica Section A. 72 (3): 406–411. doi:10.1107/S2053273316003430. ISSN 2053-2733. PMC 4850660. PMID 27126118.
Garay, Pablo G.; Martin, Osvaldo A.; Scheraga, Harold A.; Vila, Jorge A. (2016-07-21). "Detection of methylation, acetylation and glycosylation of protein residues by monitoring13C chemical-shift changes: A quantum-chemical study". PeerJ. 4: e2253. doi:10.7717/peerj.2253. ISSN 2167-8359. PMC 4963218. PMID 27547559.
Wang, Yan; Huang, Hong; Huang, Lida; Ristic, Branko (2017). "Evaluation of Bayesian source estimation methods with Prairie Grass observations and Gaussian plume model: A comparison of likelihood functions and distance measures". Atmospheric Environment. 152: 519–530. Bibcode:2017AtmEn.152..519W. doi:10.1016/j.atmosenv.2017.01.014.
MacNeil, M. Aaron; Chong-Seng, Karen M.; Pratchett, Deborah J.; Thompson, Casssandra A.; Messmer, Vanessa; Pratchett, Morgan S. (2017-03-14). "Age and Growth of An Outbreaking Acanthaster cf. solaris Population within the Great Barrier Reef". Diversity. 9 (1): 18. doi:10.3390/d9010018.
Tünnermann, Jan; Scharlau, Ingrid (2016). "Peripheral Visual Cues: Their Fate in Processing and Effects on Attention and Temporal-Order Perception". Frontiers in Psychology. 7. doi:10.3389/fpsyg.2016.01442. ISSN 1664-1078. PMC 5052275. PMID 27766086.
Graham, Nicholas A. J.; Jennings, Simon; MacNeil, M. Aaron; Mouillot, David; Wilson, Shaun K. (2015). "Predicting climate-driven regime shifts versus rebound potential in coral reefs". Nature. 518 (7537): 94–97. Bibcode:2015Natur.518...94G. doi:10.1038/nature14140. PMID 25607371.
Mascarenhas, Maya N.; Flaxman, Seth R.; Boerma, Ties; Vanderpoel, Sheryl; Stevens, Gretchen A. (2012-12-18). "National, Regional, and Global Trends in Infertility Prevalence Since 1990: A Systematic Analysis of 277 Health Surveys". PLOS Medicine. 9 (12): e1001356. doi:10.1371/journal.pmed.1001356. ISSN 1549-1676. PMC 3525527. PMID 23271957.
Cavanagh, James F; Wiecki, Thomas V; Cohen, Michael X; Figueroa, Christina M; Samanta, Johan; Sherman, Scott J; Frank, Michael J (2011). "Subthalamic nucleus stimulation reverses mediofrontal influence over decision threshold". Nature Neuroscience. 14 (11): 1462–1467. doi:10.1038/nn.2925. PMC 3394226. PMID 21946325.
Gething, Peter W.; Elyazar, Iqbal R. F.; Moyes, Catherine L.; Smith, David L.; Battle, Katherine E.; Guerra, Carlos A.; Patil, Anand P.; Tatem, Andrew J.; Howes, Rosalind E. (2012-09-06). "A Long Neglected World Malaria Map: Plasmodium vivax Endemicity in 2010". PLOS Neglected Tropical Diseases. 6 (9): e1814. doi:10.1371/journal.pntd.0001814. ISSN 1935-2735. PMC 3435256. PMID 22970336.
Pullan, Rachel L.; Smith, Jennifer L.; Jasrasaria, Rashmi; Brooker, Simon J. (2014-01-21). "Global numbers of infection and disease burden of soil transmitted helminth infections in 2010". Parasites & Vectors. 7: 37. doi:10.1186/1756-3305-7-37. ISSN 1756-3305. PMC 3905661. PMID 24447578.
Lamblin, Pascal (28 September 2017). "MILA and the future of Theano". theano-users (Mailing list). Retrieved 28 September 2017.
"Mike Lee Williams on Probabilistic Programming, Bayesian Inference, and Languages Like PyMC3". InfoQ. Retrieved 2019-01-25.
Developers, PyMC (2018-05-17). "Theano, TensorFlow and the Future of PyMC". PyMC Developers. Retrieved 2019-01-25.
Hoffman, Matthew D.; Gelman, Andrew (April 2014). "The No-U-Turn Sampler: Adaptively Setting Path Lengths in Hamiltonian Monte Carlo". Journal of Machine Learning Research. 15: pp. 1593–1623.

Kucukelbir, Alp; Ranganath, Rajesh; Blei, David M. (June 2015). "Automatic Variational Inference in Stan". 1506 (3431). arXiv:1506.03431. Bibcode:2015arXiv150603431K.