ART

In statistics, the g-prior is an objective prior for the regression coefficients of a multiple regression. It was introduced by Arnold Zellner.[1] It is a key tool in Bayes and empirical Bayes variable selection.[2][3]

Definition

Consider a data set \( (x_{1},y_{1}),\ldots ,(x_{n},y_{n}) \), where the \( x_{i} \) are Euclidean vectors and the \( y_{i} \) are scalars. The multiple regression model is formulated as

\( y_{i}=x_{i}^{\top }\beta +\varepsilon _{i}. \)

where the \( \varepsilon _{i} \) are random errors. Zellner's g-prior for \( \beta \) is a multivariate normal distribution with covariance matrix proportional to the inverse Fisher information matrix for\( \beta \) .

Assume the \( \varepsilon _{i} \) are iid normal with zero mean and variance \( \psi^{-1} \). Let X be the matrix with ith row equal to \( x_{i}^{\top } \). Then the g-prior for \( \beta \) is the multivariate normal distribution with prior mean a hyperparameter \( \beta _{0} \) and covariance matrix proportional to \( {\displaystyle \psi ^{-1}(X^{\top }X)^{-1}} \), i.e.,

\( {\displaystyle \beta |\psi \sim {\text{N}}[\beta _{0},g\psi ^{-1}(X^{\top }X)^{-1}].} \)

where g is a positive scalar parameter.

Posterior distribution of \( \beta \)

The posterior distribution of \( \beta \) is given as

\( {\displaystyle \beta |\psi ,x,y\sim {\text{N}}{\Big [}q{\hat {\beta }}+(1-q)\beta _{0},{\frac {q}{\psi }}(X^{\top }X)^{-1}{\Big ]}.} \)

where \( {\displaystyle q=g/(1+g)} \) and

\( {\displaystyle {\hat {\beta }}=(X^{\top }X)^{-1}X^{\top }y.}

is the maximum likelihood (least squares) estimator of \( \beta \) . The vector of regression coefficients \( \beta \) can be estimated by its posterior mean under the g-prior, i.e., as the weighted average of the maximum likelihood estimator and \( \beta _{0} \) ,

\( {\displaystyle {\tilde {\beta }}=q{\hat {\beta }}+(1-q)\beta _{0}.} \)

Clearly, as g →∞, the posterior mean converges to the maximum likelihood estimator.

Selection of g

Estimation of g is slightly less straightforward than estimation of \( \beta \) . A variety of methods have been proposed, including Bayes and empirical Bayes estimators.[3]

References

Zellner, A. (1986). "On Assessing Prior Distributions and Bayesian Regression Analysis with g Prior Distributions". In Goel, P.; Zellner, A. (eds.). Bayesian Inference and Decision Techniques: Essays in Honor of Bruno de Finetti. Studies in Bayesian Econometrics and Statistics. 6. New York: Elsevier. pp. 233–243. ISBN 978-0-444-87712-3.
George, E.; Foster, D. P. (2000). "Calibration and empirical Bayes variable selection". Biometrika. 87 (4): 731–747. CiteSeerX 10.1.1.18.3731. doi:10.1093/biomet/87.4.731.
Liang, F.; Paulo, R.; Molina, G.; Clyde, M. A.; Berger, J. O. (2008). "Mixtures of g priors for Bayesian variable selection". Journal of the American Statistical Association. 103 (481): 410–423. CiteSeerX 10.1.1.206.235. doi:10.1198/016214507000001337.

Undergraduate Texts in Mathematics

Graduate Texts in Mathematics

Graduate Studies in Mathematics

Mathematics Encyclopedia

World

Index

Hellenica World - Scientific Library

Retrieved from "http://en.wikipedia.org/"
All text is available under the terms of the GNU Free Documentation License