Smoothing problem (stochastic processes)
The smoothing problem is the problem of estimating an unknown probability density function recursively over time using incremental incoming measurements. It is one of the main problems defined by Norbert Wiener. A smoother is an algorithm that implements a solution to this problem, typically based on recursive Bayesian estimation. The smoothing problem is closely related to the filtering problem, both of which are studied in Bayesian smoothing theory.
A smoother is often a two-pass process, composed of forward and backward passes. Consider doing estimation about an ongoing process based on incoming observations. When new observations arrive, estimations about past needs to be updated to have a smoother estimation of the whole estimated path until now. Without a backward pass, the sequence of predictions in an online filtering algorithm does not look smooth. In other words, retrospectively, it is as if we are using future observations for improving the estimation of a point in the past, when those observations about future points become available. Note that the time of estimation can be different from the time of the point that the prediction is about. The observations about later times can be used to update and improve the estimations about earlier times. Doing so leads to smoother-looking estimations about the whole path.
Examples of smoothers
Some variants include:- Rauch–Tung–Striebel smoother
- Gaussian smoothers for non-linear state-space models.
- Particle smoothers
The confusion in terms and the relation between Filtering and Smoothing problems
The terms Smoothing and Filtering are used for four concepts that may initially be confusing: Smoothing, and Filtering .Smoothing and smoothing, despite being labelled with the same name in the English language, can mean totally different mathematical procedures. The requirements of the problems they solve are different. These concepts are distinguished by the context.
The historical reason for this confusion is that initially, Wiener's suggested "smoothing" filter was just a convolution. Later on, his proposed solutions for obtaining a smoother estimation separate developments into two distinct concepts. One was about attaining a smoother estimation by taking into account past observations, and the other was smoothing using a filter design.
Both the smoothing problem and the filtering problem are often confused with smoothing and filtering in other contexts. These names are used in the context of World War 2 with problems framed by people like Norbert Wiener. One source of confusion is that the Wiener Filter is in the form of a simple convolution. However, in Wiener's filter, two time series are given. When the filter is defined, a straightforward convolution is the answer. However, in later developments, such as Kalman filtering, the nature of filtering differs from convolution and warrants a different name.
The distinction is described in the following two senses:
1. Convolution: The smoothing in the sense of convolution is simpler. For example, moving average, low-pass filtering, convolution with a kernel, or blurring using Laplace filters in image processing. It is often a filter design problem. Especially non-stochastic and non-Bayesian signal processing, without any hidden variables.
2. Estimation: The smoothing problem uses Bayesian and state-space models to estimate the hidden state variables. This is used in the context of World War 2, defined by people like Norbert Wiener, in control theory, radar, signal detection, tracking, etc. The most common use is the Kalman Smoother used with the Kalman Filter, which was actually developed by Rauch. The procedure is called Kalman-Rauch recursion.
It is one of the main problems solved by Norbert Wiener.
Most importantly, in the Filtering problem, the information from observation up to the time of the current sample is used. In smoothing, all observation samples are used. Filtering is causal, but smoothing is batch processing of the same problem, namely, estimation of a time-series process based on serial incremental observations.
But the usual and more common smoothing and filtering do not have such a distinction because there is no distinction between hidden and observable.
The distinction between Smoothing and Filtering :
In smoothing, all observation samples are used. Filtering is causal, whereas smoothing is batch processing of the given data. Filtering is the estimation of a time-series process based on serial incremental observations.