In probability theory and statistics, the normal-Wishart distribution (or Gaussian-Wishart distribution) is a multivariate four-parameter family of continuous probability distributions. It is the conjugate prior of a multivariate normal distribution with unknown mean and precision matrix (the inverse of the covariance matrix).[1]
Definition
Suppose
\( {\displaystyle {\boldsymbol {\mu }}|{\boldsymbol {\mu }}_{0},\lambda ,{\boldsymbol {\Lambda }}\sim {\mathcal {N}}({\boldsymbol {\mu }}_{0},(\lambda {\boldsymbol {\Lambda }})^{-1})} \)
has a multivariate normal distribution with mean \( {\boldsymbol {\mu }}_{0} \) and covariance matrix \( (\lambda {\boldsymbol \Lambda })^{{-1}}, \) where
\( {\boldsymbol \Lambda }|{\mathbf {W}},\nu \sim {\mathcal {W}}({\boldsymbol \Lambda }|{\mathbf {W}},\nu ) \)
has a Wishart distribution. Then \( ({\boldsymbol \mu },{\boldsymbol \Lambda }) \) has a normal-Wishart distribution, denoted as
\( ({\boldsymbol \mu },{\boldsymbol \Lambda })\sim {\mathrm {NW}}({\boldsymbol \mu }_{0},\lambda ,{\mathbf {W}},\nu ). \)
Characterization
Probability density function
\( f({\boldsymbol \mu },{\boldsymbol \Lambda }|{\boldsymbol \mu }_{0},\lambda ,{\mathbf {W}},\nu )={\mathcal {N}}({\boldsymbol \mu }|{\boldsymbol \mu }_{0},(\lambda {\boldsymbol \Lambda })^{{-1}})\ {\mathcal {W}}({\boldsymbol \Lambda }|{\mathbf {W}},\nu ) \)
Properties
Scaling
Marginal distributions
By construction, the marginal distribution over \( {\boldsymbol {\Lambda }} \) is a Wishart distribution, and the conditional distribution over \( \boldsymbol\mu \) given \( {\boldsymbol {\Lambda }} \) is a multivariate normal distribution. The marginal distribution over \( \boldsymbol\mu \) is a multivariate t-distribution.
Posterior distribution of the parameters
After making n observations \( {\displaystyle {\boldsymbol {x}}_{1},\dots ,{\boldsymbol {x}}_{n}} \) , the posterior distribution of the parameters is
\( {\displaystyle ({\boldsymbol {\mu }},{\boldsymbol {\Lambda }})\sim \mathrm {NW} ({\boldsymbol {\mu }}_{n},\lambda _{n},\mathbf {W} _{n},\nu _{n}),} \)
where
\( {\displaystyle \lambda _{n}=\lambda +n,} \)
\( {\displaystyle {\boldsymbol {\mu }}_{n}={\frac {\lambda {\boldsymbol {\mu }}_{0}+n{\boldsymbol {\bar {x}}}}{\lambda +n}},} \)
\( {\displaystyle \nu _{n}=\nu +n,} \)
\( {\displaystyle \mathbf {W} _{n}^{-1}=\mathbf {W} ^{-1}+\sum _{i=1}^{n}({\boldsymbol {x}}_{i}-{\boldsymbol {\bar {x}}})({\boldsymbol {x}}_{i}-{\boldsymbol {\bar {x}}})^{T}+{\frac {n\lambda }{n+\lambda }}({\boldsymbol {\bar {x}}}-{\boldsymbol {\mu }}_{0})({\boldsymbol {\bar {x}}}-{\boldsymbol {\mu }}_{0})^{T}.} \) [2]
Generating normal-Wishart random variates
Generation of random variates is straightforward:
Sample \( {\boldsymbol {\Lambda }} \) from a Wishart distribution with parameters \( \mathbf {W} \) and \( \nu \)
Sample \( \boldsymbol\mu \) from a multivariate normal distribution with mean \( {\boldsymbol {\mu }}_{0} \) and variance \( (\lambda {\boldsymbol \Lambda })^{{-1}} \)
Related distributions
The normal-inverse Wishart distribution is essentially the same distribution parameterized by variance rather than precision.
The normal-gamma distribution is the one-dimensional equivalent.
The multivariate normal distribution and Wishart distribution are the component distributions out of which this distribution is made.
Notes
Bishop, Christopher M. (2006). Pattern Recognition and Machine Learning. Springer Science+Business Media. Page 690.
Cross Validated, https://stats.stackexchange.com/q/324925
References
Bishop, Christopher M. (2006). Pattern Recognition and Machine Learning. Springer Science+Business Media.
Undergraduate Texts in Mathematics
Graduate Studies in Mathematics
Hellenica World - Scientific Library
Retrieved from "http://en.wikipedia.org/"
All text is available under the terms of the GNU Free Documentation License
