Jeffreys prior

In Bayesian probability, the Jeffreys prior, named after Sir Harold Jeffreys,^[1] is a non-informative prior distribution for a parameter space; its density function is proportional to the square root of the determinant of the Fisher information matrix:

${\displaystyle p\left({\vec {\theta }}\right)\propto {\sqrt {\det {\mathcal {I}}\left({\vec {\theta }}\right)}}.\,}$

It has the key feature that it is invariant under a change of coordinates for the parameter vector ${\displaystyle {\vec {\theta }}}$. That is, the relative probability assigned to a volume of a probability space using a Jeffreys prior will be the same regardless of the parameterization used to define the Jeffreys prior. This makes it of special interest for use with scale parameters.^[2] As a concrete example, a Bernoulli distribution can be parametrized by the probability of occurrence p, or by the odds ratio. A naive uniform prior in this case is not invariant to this reparametrization, but the Jeffreys prior is.

In maximum likelihood estimation of exponential family models, penalty terms based on the Jeffreys prior were shown to reduce asymptotic bias in point estimates.^[3][4]