Chi-squared distribution


In probability theory and statistics, the -distribution with degrees of freedom is the distribution of a sum of the squares of independent standard normal random variables.
The chi-squared distribution is a special case of the gamma distribution and the univariate Wishart distribution. Specifically if then and.
The scaled chi-squared distribution is a reparametrization of the gamma distribution and the univariate Wishart distribution. Specifically if then and.
The chi-squared distribution is one of the most widely used probability distributions in inferential statistics, notably in hypothesis testing and in construction of confidence intervals. This distribution is sometimes called the central chi-squared distribution, a special case of the more general noncentral chi-squared distribution.
The chi-squared distribution is used in the common chi-squared tests for goodness of fit of an observed distribution to a theoretical one, the independence of two criteria of classification of qualitative data, and in finding the confidence interval for estimating the population standard deviation of a normal distribution from a sample standard deviation. Many other statistical tests also use this distribution, such as Friedman's analysis of variance by ranks.

Definitions

If are independent, standard normal random variables, then the sum of their squares,
is distributed according to the chi-squared distribution with degrees of freedom. This is usually denoted as
The chi-squared distribution has one parameter: a positive integer that specifies the number of degrees of freedom.

Introduction

The chi-squared distribution is used primarily in hypothesis testing, and to a lesser extent for confidence intervals for population variance when the underlying distribution is normal. Unlike more widely known distributions such as the normal distribution and the exponential distribution, the chi-squared distribution is not as often applied in the direct modeling of natural phenomena. It arises in the following hypothesis tests, among others:
It is also a component of the definition of the t-distribution and the F-distribution used in t-tests, analysis of variance, and regression analysis.
The primary reason for which the chi-squared distribution is extensively used in hypothesis testing is its relationship to the normal distribution. Many hypothesis tests use a test statistic, such as the t-statistic in a t-test. For these hypothesis tests, as the sample size,, increases, the sampling distribution of the test statistic approaches the normal distribution. Because the test statistic is asymptotically normally distributed, provided the sample size is sufficiently large, the distribution used for hypothesis testing may be approximated by a normal distribution. Testing hypotheses using a normal distribution is well understood and relatively easy. The simplest chi-squared distribution is the square of a standard normal distribution. So wherever a normal distribution could be used for a hypothesis test, a chi-squared distribution could be used.
Suppose that is a random variable sampled from the standard normal distribution, where the mean is and the variance is :. Now, consider the random variable. The distribution of the random variable is an example of a chi-squared distribution:. The subscript 1 indicates that this particular chi-squared distribution is constructed from only 1 standard normal distribution. A chi-squared distribution constructed by squaring a single standard normal distribution is said to have 1 degree of freedom. Thus, as the sample size for a hypothesis test increases, the distribution of the test statistic approaches a normal distribution. Just as extreme values of the normal distribution have low probability, extreme values of the chi-squared distribution have low probability.
An additional reason that the chi-squared distribution is widely used is that it turns up as the large sample distribution of generalized likelihood ratio tests. LRTs have several desirable properties; in particular, simple LRTs commonly provide the highest power to reject the null hypothesis and this leads also to optimality properties of generalised LRTs. However, the normal and chi-squared approximations are only valid asymptotically. For this reason, it is preferable to use the t distribution rather than the normal approximation or the chi-squared approximation for a small sample size. Similarly, in analyses of contingency tables, the chi-squared approximation will be poor for a small sample size, and it is preferable to use Fisher's exact test. Ramsey shows that the exact binomial test is always more powerful than the normal approximation.
Lancaster shows the connections among the binomial, normal, and chi-squared distributions, as follows. De Moivre and Laplace established that a binomial distribution could be approximated by a normal distribution. Specifically they showed the asymptotic normality of the random variable
where is the observed number of successes in trials, where the probability of success is, and.
Squaring both sides of the equation gives
Using,, and, this equation can be rewritten as
The expression on the right is of the form that Karl Pearson would generalize to the form
where
  • = Pearson's cumulative test statistic, which asymptotically approaches a distribution;
  • = the number of observations of type ;
  • = the expected frequency of type, asserted by the null hypothesis that the fraction of type in the population is ; and
  • = the number of cells in the table.
In the case of a binomial outcome, the binomial distribution may be approximated by a normal distribution. Because the square of a standard normal distribution is the chi-squared distribution with one degree of freedom, the probability of a result such as 1 heads in 10 trials can be approximated either by using the normal distribution directly, or the chi-squared distribution for the normalised, squared difference between observed and expected value. However, many problems involve more than the two possible outcomes of a binomial, and instead require 3 or more categories, which leads to the multinomial distribution. Just as de Moivre and Laplace sought for and found the normal approximation to the binomial, Pearson sought for and found a degenerate multivariate normal approximation to the multinomial distribution. Pearson showed that the chi-squared distribution arose from such a multivariate normal approximation to the multinomial distribution, taking careful account of the statistical dependence between numbers of observations in different categories.

Probability density function

The probability density function of the chi-squared distribution is
where denotes the gamma function, which has closed-form values for integer.
For derivations of the pdf in the cases of one, two and degrees of freedom, see Proofs related to chi-squared distribution.

Cumulative distribution function

Its cumulative distribution function is:
where is the lower incomplete gamma function and is the regularized gamma function.
In a special case of this function has the simple form:
which can be easily derived by integrating directly. The integer recurrence of the gamma function makes it easy to compute for other small, even.
Tables of the chi-squared cumulative distribution function are widely available and the function is included in many spreadsheets and all statistical packages.
Letting, Chernoff bounds on the lower and upper tails of the CDF may be obtained. For the cases when :
The tail bound for the cases when, similarly, is
For another approximation for the CDF modeled after the cube of a Gaussian, see under Noncentral chi-squared distribution.

Properties

Cochran's theorem

The following is a special case of Cochran's theorem.
Theorem. If are independent identically distributed, standard normal random variables, then
where
Proof. Let be a vector of independent normally distributed random variables,
and their average.
Then
where is the identity matrix and the all ones vector.
has one eigenvector with eigenvalue,
and eigenvectors with eigenvalue,
which can be chosen so that is an orthogonal matrix.
Since also,
we have
which proves the claim.

Additivity

It follows from the definition of the chi-squared distribution that the sum of independent chi-squared variables is also chi-squared distributed. Specifically, if are independent chi-squared variables with, degrees of freedom, respectively, then is chi-squared distributed with degrees of freedom.

Sample mean

The sample mean of i.i.d. chi-squared variables of degree is distributed according to a gamma distribution with shape and scale parameters:
[|Asymptotically], given that for a shape parameter going to infinity, a Gamma distribution converges towards a normal distribution with expectation and variance the sample mean converges towards:
Note that we would have obtained the same result invoking instead the central limit theorem, noting that for each chi-squared variable of degree the expectation is and its variance .

Entropy

The differential entropy is given by
where is the Digamma function.
The chi-squared distribution is the maximum entropy probability distribution for a random variate for which and are fixed. Since the chi-squared is in the family of gamma distributions, this can be derived by substituting appropriate values in the Expectation of the log moment of gamma. For derivation from more basic principles, see the derivation in moment-generating function of the sufficient statistic.