Logistic function


A logistic function or logistic curve is a common S-shaped curve with the equation
where
  • is the carrying capacity, the supremum of the values of the function;
  • is the logistic growth rate, the steepness of the curve; and
  • is the value of the function's midpoint.
The logistic function has domain the real numbers, the limit as is 0, and the limit as is.
The exponential function with negated argument is used to define the standard logistic function where, which has the equation
and is sometimes simply called the sigmoid function. It is also sometimes called the expit, being the inverse function of the logit.
The logistic function finds applications in a range of fields, including biology, biomathematics, chemistry, demography, economics, geoscience, mathematical psychology, probability, sociology, political science, linguistics, statistics, and artificial neural networks. There are various [|generalizations], depending on the field.

History

The logistic function was introduced in a series of three papers by Pierre François Verhulst between 1838 and 1847, who devised it as a model of population growth by adjusting the exponential growth model, under the guidance of Adolphe Quetelet. Verhulst first devised the function in the mid 1830s, publishing a brief note in 1838, then presented an expanded analysis and named the function in 1844 ; the third paper adjusted the correction term in his model of Belgian population growth.
The initial stage of growth is approximately exponential ; then, as saturation begins, the growth slows to linear, and at maturity, growth approaches the limit with an exponentially decaying gap, like the initial stage in reverse.
Verhulst did not explain the choice of the term "logistic", but it is presumably in contrast to the logarithmic curve, and by analogy with arithmetic and geometric. His growth model is preceded by a discussion of arithmetic growth and geometric growth, and thus "logistic growth" is presumably named by analogy, logistic being from, a traditional division of Greek mathematics.
As a word derived from ancient Greek mathematical terms,
the name of this function is unrelated to the military and management term logistics, which is instead from "lodgings", though some believe the Greek term also influenced logistics; see for details.

Mathematical properties

The is the logistic function with parameters,,, which yields
In practice, due to the nature of the exponential function, it is often sufficient to compute the standard logistic function for over a small range of real numbers, such as a range contained in , as it quickly converges very close to its saturation values of 0 and 1.

Symmetries

The logistic function has the symmetry property that
This reflects that the growth from 0 when is small is symmetric with the decay of the gap to the limit when is large.
Further, is an odd function.
The sum of the logistic function and its reflection about the vertical axis,, is
The logistic function is thus rotationally symmetrical about the point.

Inverse function

The logistic function is the inverse of the natural logit function
and so converts the logarithm of odds into a probability.
The conversion from the log-likelihood ratio of two alternatives also takes the form of a logistic curve.

Hyperbolic tangent

The logistic function is an offset and scaled hyperbolic tangent function:
or
This follows from
The hyperbolic-tangent relationship leads to another form for the logistic function's derivative:
which ties the logistic function into the logistic distribution.
Geometrically, the hyperbolic tangent function is the hyperbolic angle on the unit hyperbola, which factors as, and thus has asymptotes the lines through the origin with slope and with slope, and vertex at corresponding to the range and midpoint of tanh. Analogously, the logistic function can be viewed as the hyperbolic angle on the hyperbola, which factors as, and thus has asymptotes the lines through the origin with slope and with slope, and vertex at, corresponding to the range and midpoint of the logistic function.
Parametrically, hyperbolic cosine and hyperbolic sine give coordinates on the unit hyperbola:, with quotient the hyperbolic tangent. Similarly, parametrizes the hyperbola, with quotient the logistic function. These correspond to linear transformations of the hyperbola, with parametrization : the parametrization of the hyperbola for the logistic function corresponds to and the linear transformation, while the parametrization of the unit hyperbola corresponds to the linear transformation.

Derivative

The standard logistic function has an easily calculated derivative. The derivative is known as the density of the logistic distribution:
from which all higher derivatives can be derived algebraically. For example,.
The logistic distribution is a location–scale family, which corresponds to parameters of the logistic function. If is fixed, then the midpoint is the location and the slope is the scale.

Integral

Conversely, its antiderivative can be computed by the substitution, since
so
In artificial neural networks, this is known as the softplus function and is a smooth approximation of the ramp function, just as the logistic function is a smooth approximation of the Heaviside step function.

Taylor series

The standard logistic function is analytic on the whole real line since, where, and, are analytic on their domains, and the composition of analytic functions is again analytic.
A formula for the nth derivative of the standard logistic function is
therefore its Taylor series about the point is

Logistic differential equation

The unique standard logistic function is the solution of the simple first-order non-linear ordinary differential equation
with boundary condition. This equation is the continuous version of the logistic map. Note that the reciprocal logistic function is solution to a simple first-order linear ordinary differential equation.
The qualitative behavior is easily understood in terms of the phase line: the derivative is 0 when the function is 1; and the derivative is positive for between 0 and 1, and negative for above 1 or less than 0. This yields an unstable equilibrium at 0 and a stable equilibrium at 1, and thus for any function value greater than 0 and less than 1, it grows to 1.
The logistic equation is a special case of the Bernoulli differential equation and has the following solution:
Choosing the constant of integration gives the other well known form of the definition of the logistic curve:
More quantitatively, as can be seen from the analytical solution, the logistic curve shows early exponential growth for negative argument, which reaches to linear growth of slope 1/4 for an argument near 0, then approaches 1 with an exponentially decaying gap.
The differential equation derived above is a special case of a general differential equation that only models the sigmoid function for. In many modeling applications, the more general form
can be desirable. Its solution is the shifted and scaled sigmoid function.

Probabilistic interpretation

When the capacity, the value of the logistic function is in the range and can be interpreted as a probability. In more detail, can be interpreted as the probability of one of two alternatives ; the two alternatives are complementary, so the probability of the other alternative is and. The two alternatives are coded as 1 and 0, corresponding to the limiting values as.
In this interpretation the input is the log-odds for the first alternative, and so is the odds for the first alternative. Given odds for an event of , the probability is the ratio of "for" over "for plus against",. We see that the logistic function,, is the probability of the first alternative.
Conversely, is the log-odds against the second alternative, is the log-odds for the second alternative, is the odds for the second alternative, and is the probability of the second alternative.
This can be framed more symmetrically in terms of two inputs, and, which then generalizes naturally to more than two alternatives. Given two real number inputs, and, interpreted as logits, their difference is the log-odds for option 1, is the odds,
is the probability of option 1, and similarly is the probability of option 0.
This form immediately generalizes to more alternatives as the softmax function, which is a vector-valued function whose -th coordinate is.
More subtly, the symmetric form emphasizes interpreting the input as and thus relative to some reference point, implicitly to. Notably, the softmax function is invariant under adding a constant to all the logits, which corresponds to the difference being the log-odds for option against option, but the individual logits not being log-odds on their own. Often one of the options is used as a reference, and its value fixed as, so the other logits are interpreted as odds versus this reference. This is generally done with the first alternative, hence the choice of numbering:, and then is the log-odds for option against option. Since, this yields the term in many expressions for the logistic function and generalizations.

Generalizations

In growth modeling, numerous generalizations exist, including the generalized logistic curve, the Gompertz function, the cumulative distribution function of the shifted Gompertz distribution, and the hyperbolastic function of type I.
In statistics, where the logistic function is interpreted as the probability of one of two alternatives, the generalization to three or more alternatives is the softmax function, which is vector-valued, as it gives the probability of each alternative.