Chapman–Robbins bound

In statistics, the Chapman–Robbins bound or Hammersley–Chapman–Robbins bound is a lower bound on the variance of estimators of a deterministic parameter. It is a generalization of the Cramér–Rao bound; compared to the Cramér–Rao bound, it is both tighter and applicable to a wider range of problems. However, it is usually more difficult to compute.
The bound was independently discovered by John Hammersley in 1950, and by Douglas Chapman and Herbert Robbins in 1951.

Statement

Let be the set of parameters for a family of probability distributions on.
For any two, let be the -divergence from to. Then:
A generalization to the multivariable case is:

Proof

By the variational representation of chi-squared divergence:
Plug in, to obtain: Switch the denominator and the left side and take supremum over to obtain the single-variate case. For the multivariate case, we define for any. Then plug in in the variational representation to obtain: Take supremum over, using the linear algebra fact that, we obtain the multivariate case.

Relation to Cramér–Rao bound

Usually, is the sample space of independent draws of a -valued random variable with distribution from a by parameterized family of probability distributions, is its -fold product measure, and is an estimator of. Then, for, the expression inside the supremum in the Chapman–Robbins bound converges to the Cramér–Rao bound of when, assuming the regularity conditions of the Cramér–Rao bound hold. This implies that, when both bounds exist, the Chapman–Robbins version is always at least as tight as the Cramér–Rao bound; in many cases, it is substantially tighter.
The Chapman–Robbins bound also holds under much weaker regularity conditions. For example, no assumption is made regarding differentiability of the probability density function p of. When p is non-differentiable, the Fisher information is not defined, and hence the Cramér–Rao bound does not exist.