Finite difference

A finite difference is a mathematical expression of the form. Finite differences are often used as approximations of derivatives, such as in numerical differentiation.
The difference operator, commonly denoted, is the operator that maps a function to the function defined by
A difference equation is a functional equation that involves the finite difference operator in the same way as a differential equation involves derivatives. There are many similarities between difference equations and differential equations. Certain recurrence relations can be written as difference equations by replacing iteration notation with finite differences.
In numerical analysis, finite differences are widely used for [|approximating derivatives], and the term "finite difference" is often used as an abbreviation of "finite difference approximation of derivatives".
Finite differences were introduced by Brook Taylor in 1715 and have also been studied as abstract self-standing mathematical objects in works by George Boole, L. M. Milne-Thomson, and . Finite differences trace their origins back to one of Jost Bürgi's algorithms and work by others including Isaac Newton. The formal calculus of finite differences can be viewed as an alternative to the calculus of infinitesimals.

Basic types

Three basic types are commonly considered: forward, backward, and central finite differences.
A ', denoted of a function is a function defined as
Depending on the application, the spacing may be variable or constant. When not specified, the default value for is 1; that is,
A ' uses the function values at and , instead of the values at
Finally, the is given by

Relation with derivatives

The approximation of derivatives by finite differences plays a central role in finite difference methods for the numerical solution of differential equations, especially boundary value problems.
The derivative of a function at a point is defined by the limit
If has a fixed value instead of approaching zero, then the right-hand side of the above equation would be written
Hence, the forward difference divided by approximates the derivative when is small. The error in this approximation can be derived from Taylor's theorem. Assuming that is twice differentiable, we have
The same formula holds for the backward difference:
However, the central difference yields a more accurate approximation. If is three times differentiable,
The main problem with the central difference method, however, is that oscillating functions can yield zero derivative. If for odd, and for even, then if it is calculated with the central difference scheme. This is particularly troublesome if the domain of is discrete. See also Symmetric derivative.
Authors for whom finite differences mean finite difference approximations define the forward/backward/central differences as the quotients given in this section.

Higher-order differences

In an analogous way, one can obtain finite difference approximations to higher order derivatives and differential operators. For example, by using the above central difference formula for and and applying a central difference formula for the derivative of at, we obtain the central difference approximation of the second derivative of
;Second-order central :
Similarly we can apply other differencing formulas in a recursive manner.
;Second-order forward :
;Second-order backward :
More generally, the -th order forward, backward, and central differences are given by, respectively,
;Forward :
;Backward :
;Central :
These equations use binomial coefficients after the summation sign shown as Each row of Pascal's triangle provides the coefficient for each value of
Note that the central difference will, for odd, have multiplied by non-integers. This is often a problem because it amounts to changing the interval of discretization. The problem may be remedied substituting the average of and
Forward differences applied to a sequence are sometimes called the binomial transform of the sequence, and have a number of interesting combinatorial properties. Forward differences may be evaluated using the Nörlund-Rice integral. The integral representation for these types of series is interesting, because the integral can often be evaluated using asymptotic expansion or saddle-point techniques; by contrast, the forward difference series can be extremely hard to evaluate numerically, because the binomial coefficients grow rapidly for large.
The relationship of these higher-order differences with the respective derivatives is straightforward,
Higher-order differences can also be used to construct better approximations. As mentioned above, the first-order difference approximates the first-order derivative up to a term of order. However, the combination
approximates up to a term of order. This can be proven by expanding the above expression in Taylor series, or by using the calculus of finite differences, explained below.
If necessary, the finite difference can be centered about any point by mixing forward, backward, and central differences.
Sometimes, the low order derivatives of a function may be analytically known, but high order derivatives are not. In these cases, the high order derivatives can be approximated by finite difference of low order derivatives, which is often more accurate and numerically more stable than finite difference of the function itself. This is sometimes called seminumerical differentiation. For example, when the first order derivative is available but the second order derivative is not, the latter can be approximated by second-order central difference of :

Polynomials

For a given polynomial of degree, expressed in the function, with real numbers and and lower order terms marked as :
After pairwise differences, the following result can be achieved, where is a real number marking the arithmetic difference:
Only the coefficient of the highest-order term remains. As this result is constant with respect to, any further pairwise differences will have the value.

Inductive proof

Base case

Let be a polynomial of degree :
This proves it for the base case.

Inductive step

Let be a polynomial of degree where and the coefficient of the highest-order term be. Assuming the following holds true for all polynomials of degree :
Let be a polynomial of degree. With one pairwise difference:
As, this results in a polynomial of degree, with as the coefficient of the highest-order term. Given the assumption above and pairwise differences, it can be found that:
This completes the proof.

Application

This identity can be used to find the lowest-degree polynomial that intercepts a number of points where the difference on the x-axis from one point to the next is a constant. For example, given the following points:

x	y
1	4
4	109
7	772
10	2641
13	6364

We can use a differences table, where for all cells to the right of the first, the following relation to the cells in the column immediately to the left exists for a cell, with the top-leftmost cell being at coordinate :
To find the first term, the following table can be used:


1	4	-	-	-
4	109	105	-	-
7	772	663	558	-
10	2641	1869	1206	648
13	6364	3723	1854	648

This arrives at a constant. The arithmetic difference is, as established above. Given the number of pairwise differences needed to reach the constant, it can be surmised this is a polynomial of degree. Thus, using the identity above:
Solving for, it can be found to have the value. Thus, the first term of the polynomial is.
Then, subtracting out the first term, which lowers the polynomial's degree, and finding the finite difference again:


1	-	-
4	−147	-
7	−453	−306
10	−759	−306
13	−1065	−306

Here, the constant is achieved after only two pairwise differences, thus the following result:
Solving for, which is, the polynomial's second term is.
Moving on to the next term, by subtracting out the second term:


1		-
4		108
7		108
10		108
13		108

Thus the constant is achieved after only one pairwise difference:
It can be found that and thus the third term of the polynomial is. Subtracting out the third term:


1
4
7
10
13

Without any pairwise differences, it is found that the 4th and final term of the polynomial is the constant. Thus, the lowest-degree polynomial intercepting all the points in the first table is found:

Arbitrarily sized kernels

Using linear algebra one can construct finite difference approximations which utilize an arbitrary number of points to the left and a number of points to the right of the evaluation point, for any order derivative. This involves solving a linear system such that the Taylor expansion of the sum of those points around the evaluation point best approximates the Taylor expansion of the desired derivative. Such formulas can be represented graphically on a hexagonal or diamond-shaped grid.
This is useful for differentiating a function on a grid, where, as one approaches the edge of the grid, one must sample fewer and fewer points on one side.
Finite difference approximations for non-standard stencils given an arbitrary stencil and a desired derivative order may be constructed.