Relationships between statistics and physics often provide a deeper understanding that can lead new or improved algorithmic approaches to solving statistics problems. It is well known that the negative logarithm of a probability distribution is analogous to a physical potential. I describe a novel approach, borrowed from physics, to estimating specified components of the covariance matrix C. In making an analogy of the minus-log-posterior, phi, as a physical potential, the notion is to determine the displacement of the minimizer of phi under the influence of an external force. It is easy to show that the displacement in the parameters is proportional to the product C f, where f is the force applied to the system. This force represents a linear combination of the parameters about which we want to estimate the uncertainty. The variance in the direction of f may be estimated as the product of f with the displacement. Furthermore, the covariance between f and another linear combination of parameters may similarly be estimated. This approach to uncertainty estimation is most useful in situations in which: a) the standard techniques for variance estimation are costly, b) it is relatively easy to find phi, and c) one is interested in the uncertainty with respect to one, or a few directions in the parameter space, not the full covariance matrix. The usefulness of this new technique is demonstrated with examples ranging from simple to complex.
Keywords: covariance estimation, probability potential, external force, posterior stiffness
Get full paper (pdf, 331 KB)
Return to publication list
Send e-mail to author at kmh@hansonhub.com