Spatiotemporal reservoir resampling

Spatiotemporal reservoir resampling, commonly known as ReSTIR, is a collection of computer graphics techniques for generating and reusing samples during rendering. It was developed primarily to allow more realistic lighting in real-time rendering, because relatively few rays can be traced per pixel while maintaining an acceptable frame rate. It can also be used to speed up off-line path tracing.
The first ReSTIR paper, published in 2020, provided algorithms for direct lighting, allowing scenes containing thousands of lights to be rendered in real time on a high-end GPU. Researchers later proposed versions for rendering indirect lighting and built up a framework of mathematical concepts and notation conventions that help analyze such algorithms.
A major focus of this work is removing or reducing the bias that could be introduced when samples from other pixels or frames are reused—or selectively allowing some bias in order to speed up rendering and reduce variance. Versions for path tracing apply transformations called shift mappings to samples, typically reusing parts of paths closer to the light and modifying the portion closer to the camera.
ReSTIR-related papers and talks have been presented every year at the SIGGRAPH conference since 2020.
One of the first games to incorporate ReSTIR into its rendering was Cyberpunk 2077.

Overview and motivation

According to Chris Wyman, one of the co-authors of the original paper, although developers commonly thought that bias was acceptable for real-time rendering, end users are well-aware of the artifacts caused by bias and many have a negative opinion of common sample-reuse techniques such as temporal anti-aliasing, which may cause "ghosting" when the camera moves, and denoising, which causes blurring and other artifacts.
ReSTIR techniques can reduce or avoid these types of bias by reusing samples of the set of possible paths taken by light to reach the camera, instead of reusing rendered pixel color values. While other techniques reuse samples in a generic post-processing step, ReSTIR passes can test for shadowing, and reused samples are converted into pixel color values by rendering code that takes the characteristics of different materials into account. However the output of ReSTIR is noisy, and a denoising pass is typically still used.
Stochastic ray tracing techniques such as path tracing need to average multiple samples in order to render a visually acceptable image. When using a simple unbiased renderer based on Monte Carlo integration, halving the deviation of the result requires multiplying the number of samples by four, meaning that a rapidly increasingly number of samples is needed to improve quality, Standard ways to mitigate this problem include importance sampling, and quasi-Monte Carlo integration. ReSTIR offers a solution that multiplies the effective number of samples while tracing a fixed number of additional rays per frame.
Temporal reuse multiplies the effective sample count by the number of frames rendered. Spatial reuse multiplies the effective count by the number of neighboring pixels examined. These two types of reuse can be combined, allowing spatial reuse to be applied recursively, which appears to offer an exponentially increasing effective sample count, however this is quickly limited by the size of the neighborhood used for spatial reuse. Spatial reuse is also potentially less effective near shadow and object edges, especially for objects with fine geometric detail, and temporal reuse is limited by movement of the camera and scene elements.

Variations

Many variations of ReSTIR have been proposed that generalize or improve the original technique, specialize it for particular types of illumination or other visual effects, or allow incorporation into rendering algorithms other than standard path tracing. Some published versions are listed below.

Name	Paper	Applicability
Resampled Importance Sampling	Talbot et al. 2005 Importance Resampling for Global Illumination	light sampling, [Bidirectional reflectance Probability distribution\|distribution function\|BRDF] sampling
ReSTIR, ReSTIR DI	Bitterli et al. 2020 Spatiotemporal reservoir resampling for real-time ray tracing with dynamic direct lighting	direct illumination
ReSTIR GI	Ouyang et al. 2021 ReSTIR GI: Effective Path Resampling for Real-Time Path Tracing	indirect illumination; optimized for diffuse surfaces
ReGIR	Boksansky et al. 2021 Rendering many lights with grid-based reservoirs	light sampling during path tracing
Volumetric ReSTIR	Lin et al. 2021 Fast Volume Rendering with Spatiotemporal Reservoir Resampling	"participating media" such as clouds and smoke
ReSTIR PT, Generalized Resampled Importance Sampling	Lin et al. 2022 Generalized resampled importance sampling: foundations of ReSTIR	general path tracing; reflective surfaces; some caustics
Suffix ReSTIR	Kettunen et al. 2023 Conditional Resampled Importance Sampling and ReSTIR	general path tracing; light reflecting from specular surfaces; fast camera motion
ReSTIR FG	Kern et al. 2024 ReSTIR FG: Real-Time Reservoir Resampled Photon Final Gathering	caustics, glass-encased objects, general indirect illumination
Area ReSTIR	Zhang et al. 2024 Area ReSTIR: Resampling for Real-Time Defocus and Antialiasing	general path tracing; depth of field and fine structures such as hair and foliage
Reservoir Splatting	Liu et al. 2025 Reservoir Splatting for Temporal Path Resampling and Motion Blur	general path tracing; moving objects and motion blur

Algorithms

Basic algorithm

ReSTIR uses a combination of resampled importance sampling and weighted reservoir sampling which the authors call streaming RIS. RIS processes samples from an initial probability distribution and generates samples in a new probability distribution. WRS allows this to be done while storing only a small number of samples in memory, which is especially helpful on a GPU. Information about the samples is stored in a data structure called a reservoir. WRS also allows samples from multiple reservoirs to be combined into a single reservoir; this is crucial for sample reuse.
Each pixel has a reservoir, typically containing only a single sample when ReSTIR is used for real-time rendering. The reservoir is typically initialized to a sample drawn using a simple method and is then updated by RIS steps and by reservoir merging, so that the pixel value produced by shading using the sample currently in the reservoir, times the weight for the sample, is always an unbiased estimate of the correct pixel value. If appropriate resampling steps are used, the variance of this estimate decreases with each step.
A possible sequence of steps performed for each frame, suitable for computing unbiased direct illumination is:

Perform reservoir resampling by drawing multiple light samples and using streaming RIS to choose one, using probabilities based on a target function, e.g. the luminance of the sample's contribution to the pixel. A weight is also computed for the sample. Typically, a single visibility check is performed here, after choosing a sample, setting the weight to 0 if the light is shadowed. Resampling ensures that the expected value of the weight times the sample brightness is the correct value for the pixel.
For each pixel, merge the sample from the previous frame into the current reservoir. Multiple importance sampling weights are used to avoid bias due to the fact that the samples in the previous frame's reservoirs may have a different target probability distribution if the objects, lights, or camera have moved.
For each pixel, choose one or more neighboring pixels and merge their samples into the current pixel's reservoir. Multiple importance sampling weights are used to avoid bias due to the fact that the samples in each pixel's reservoir have a different target probability distribution. Because computing unbiased MIS weights requires tracing additional rays, real-time rendering often uses only a single neighboring pixel.
Use the sample in each pixel's reservoir, along with its weight, to determine the color of the pixel for the current frame. Alternatively, multiple samples examined during the preceding steps may be averaged and used to shade the pixel instead.

For direct lighting, the initial samples used in step 1 are typically drawn by importance sampling from the set of lights in a scene. The algorithm above draws many lower-quality light samples using a fast method, without considering visibility, and chooses one using streaming RIS. Visibility is then tested for the final chosen sample. Considering visibility for each sample drawn would require tracing 32 rays, which would make it much more expensive. The intent is to reduce the number of rays traced, relying on the sample reuse in steps 2 and 3 to make up for the loss of quality caused by rejecting many of the rays due to shadowing. A large part of the initial efforts to optimize ReSTIR went into reducing the cost of randomly sampling the lights. Glossy surfaces may require a larger number of samples, and combining light sampling with BRDF sampling may increase quality.
Step 2 is sometimes skipped for off-line rendering, and the output of multiple repetitions of initial sampling and spatial reuse is averaged instead; this helps avoids artifacts due to correlations. Step 3 may be repeated multiple times in a single frame.
The neighboring pixels used in step 3 are chosen randomly, typically from a circle or square around the pixel, e.g. with a radius of 10–30 pixels. Heuristics may be used to choose neighboring pixels that are more likely to have similar surface and lighting characteristics to the target pixel, provided that these heuristics do not examine the samples in the reservoirs.
In some variations, the neighbor samples used in step 3 come from the previous frame, which can help improve performance on GPUs.

Unbiased contribution weights

Importance sampling, a technique for improving the efficiency of Monte Carlo integration, requires knowing the probability of each sample used. These probabilities cannot usually be determined for the optimal probability density function for importance sampling, and so some other PDF is used instead. Some PDFs can be sampled directly, while others require rejection sampling or another method.
A method called importance resampling provides an alternative way to generate samples from a desired PDF, taking samples from an input PDF and generating new samples with a different PDF by randomly choosing between the input samples. It can use an "unnormalized PDF", which is a function that is proportional to the actual PDF but multiplied by some unknown scaling factor. This is potentially helpful for importance sampling, where the optimal PDF is proportional to the function being integrated, but the normalization factor required to turn this function into a proper PDF is unknown. By itself, this method is biased, but if the generated samples are used for importance sampling in a particular way, the result will be unbiased, and it is not necessary to know the normalized probability values, so the optimal PDF can be used. This combined technique is called resampled importance sampling.
In ReSTIR, this method is generalized to support both sample generation and sample reuse. The conventional importance sampling weight is replaced by a weight that is a random variable whose expected value is. This weight is termed an unbiased contribution weight. These values can also be used as a substitute for when computing multiple importance sampling weights, in order to combine samples from different distributions.
The error in unbiased contribution weights is a source of variance, but the variance of the final rendered pixel values may be reduced by sample reuse and by the use of an improved PDF during initial light sampling, which is made possible by the use of these weights.

Confidence weights

Typically, new samples are introduced in each frame. These new samples are merged with samples temporally reused from the previous frame. However, if the camera and scene objects have not moved, the samples from the previous frame will be higher quality and should be replaced by new samples less frequently. Confidence weights are used to track how many effective samples contributed to each reservoir, and these weights are used to favor the sample already in the reservoir during reservoir merging. If temporal reuse is not possible for a pixel, the confidence weight is set to zero and the reservoir is always re-initialized to a new sample. The confidence weight is increased by 1 after merging a new sample into the reservoir.
During temporal reuse, samples from different reservoirs are merged, and their confidence weights are typically added to get the confidence weight for the chosen sample. However this will cause exponentially increasing confidence weights, which would soon prevent any new samples from being used, and so the value is usually capped at a value such as 20.

Backprojection

As in temporal anti-aliasing, temporal reuse is improved for moving images by reusing samples from the pixel in the previous frame that most closely matches the current pixel. Motion vectors are typically used to determine this pixel. Unlike in TAA, an individual sample must be selected, and interpolation is not used. Area ReSTIR and Reservoir Splatting enhance this approach, allowing ReSTIR to perform antialiasing and motion blur.

Biased versions

The main cost of the above algorithm is usually tracing rays to check for visibility of lights. Some of these shadowing checks may be skipped at the cost of introducing bias. Skipping visibility checks prevents correct MIS weights from being calculated, however since the use of correct MIS weights tends to increase the variance of samples in the scene the biased version may be preferable anyway. If visibility checks are skipped during spatial reuse, the biased version is likely to produce darker pixels around the edges of objects, and around shadow edges.
Multiple approaches for avoiding tracing additional rays have been explored. One possibility, considered in early versions of ReSTIR, is to not consider visibility at all during resampling and reuse, and test visibility only when using samples for rendering. This can be unbiased, but it produces much noisier output when there are shadows because the probability density function of the samples is much further from the ideal PDF for importance sampling. Compared to this method, a strategy called visibility reuse provides a relatively cheap way to get a large increase in quality. Using heuristics to select neighboring pixels that are likely to have similar lighting helps mitigate the bias caused by visibility reuse.

Generalized (path tracing) algorithm

A limitation of the efficiency of the basic ReSTIR algorithm is that the probability density function used for samples in the previous frame, or for neighboring pixels, may be different than the desired PDF for the pixel in the current frame. These different contexts, where the PDF may be different, are referred to as domains. In some cases, it is possible to increase the benefit of sample reuse by adjusting the samples during reuse, so that the resulting PDF is likely to be a better fit for the current pixel. This is especially valuable when rendering specular reflection, where a small change in position or surface normal can cause a large change in the distribution of reflected rays. Any deterministic, invertible transformations of samples are allowed provided the probability values can be correctly adjusted when performing the transformation.
Shifting of samples can be handled by finding an appropriate transformation between the samples used in different domains, and multiplying probabilities by the Jacobian determinant of the transformation function when moving between domains. Probability value adjustments are required when computing multiple importance sampling weights to remove bias during spatial and temporal reuse.
To handle cases such as when a ray is specularly reflected or refracted multiple times before hitting the point being rendered, a variety of transformations of paths are used. These transformations are called shift mappings. The mappings most commonly used for ReSTIR are listed below:reconnection shift - produces a path that has a different ray from camera to first intersection point, and from the first intersection to the second intersection, but keeps remaining path segments leading to the light source unchanged. The modified segments of the path are deterministic functions of the view ray for the pixel, and the reconnection vertex. ReSTIR DI uses a simple case of the reconnection shift where the reconnection vertex is a point on a light source. The reconnection shift typically works well for diffuse surfaces.half-vector copy - uses a deterministic way to modify segments close to the camera, allowing the reconnection shift to be used more effectively with surfaces that are highly reflective.random replay - uses the same random numbers as were used when generating the original path, but uses them to sample from different PDFs. A single "seed" value for the random number generator determines the entire path, reducing storage requirements for the sample, however paths tend to diverge further away from the camera which can make the technique less effective.hybrid shift - uses random replay for one or more segments close to the camera, along with a deterministic segment to connect to the reconnection vertex. The reconnection vertex is usually the first diffuse reflection point found along the path. The remainder of the path is unchanged. This typically improves upon the reconnection shift for glossy surfaces. It tends to produce results similar to the half-vector copy shift, but is more widely applicable.
For ReSTIR it is important to avoid needing to explicitly store a path as a sample in a ReSTIR reservoir. For the random replay shift, only the seed for the random number generator needs to be stored. For other shifts, the location of the reconnection vertex is stored, and perhaps the amount and direction of incoming light.

Volume rendering

The general path tracing version of ReSTIR described above was primarily designed for surface intersections, however it can support paths that intersect participating media by integrating over a domain of paths with vertices both on surfaces and in regions of space. Enhancements such as alternative shift mappings and sampling approaches may be required to make it work efficiently.
The Volumetric ReSTIR algorithm provides methods for real-time volume rendering of objects such as clouds and explosions. Resampled importance sampling is used to sample paths through the media, allowing cheaper approximations when calculating resampling weights for candidate samples, with expensive unbiased evaluation for the chosen samples only. The backprojection approach typically used in ReSTIR and TAA may not work well for volume rendering, and so a specialized reprojection method is used for temporal reuse. Two possible shift mappings for spatial reuse of samples were studied.

Anti-aliasing, depth of field, and motion blur

Earlier ReSTIR implementations typically use a single camera ray direction for each pixel, calculated from the coordinates of the pixel on the image plane. If the camera is moving, this direction changes smoothly from frame to frame. A pinhole lens is typically used. In contrast, off-line path tracers usually sample random points on the image plane to avoid spatial aliasing, sample random points on the aperture to simulate depth of field, and sample random points in time to render motion blur. Similarly, temporal anti-aliasing, used for real-time rendering, changes the image plane sample locations from frame to frame, usually with a non-random pattern. These types of sampling can be used with ReSTIR, but to avoid bias they must be taken into account during spatiotemporal reuse computations, and they may cause less effective sample reuse, particularly for detailed objects.
To add support for anti-aliasing, the Area ReSTIR algorithm stores an image plane location for each sample, which is used during multiple importance sampling weight calculations and shift mapping computations. The algorithm searches a neighborhood of pixels in the previous frame to find candidate samples for temporal reuse, since multiple pixels might have a sample whose forward-projected image plane location falls inside the current pixel's filter kernel. To support depth of field, shift mappings called lens vertex copy and primary hit reconnection are combined using MIS.
The fractional backprojection approach for temporal reuse in Area ReSTIR only searches a small neighborhood of pixels, and so it does not work well for images with significant blur caused by motion and depth of field, when relevant samples could be found in a much larger area of the previous frame. The Reservoir Splatting algorithm instead "scatters" each sample to one or more pixels in the following frame. There will likely be some pixels that do not receive any samples during scattering, and the gathering technique from Area ReSTIR is used to fill in these "holes".

Challenges and limitations

Computational cost

Various implementation issues may limit the applicability of ReSTIR for real-time rendering except on high-end GPUs. These problems may be addressed by GPU performance improvements.

For unbiased rendering, at least two additional rays must be traced for each pixel during each spatial and each temporal reuse step, and one ray must be traced to check visibility when introducing new samples. These figures are for direct illumination only. At the time ReSTIR was introduced, this made unbiased rendering at full resolution infeasible for real-time rendering in games, because typically only a single ray could be traced per pixel while maintaining an acceptable frame rate.
Unbiased temporal reuse requires tracing rays using scene data from the previous frame, which may increase memory use, and may make ReSTIR more difficult to integrate into game engines.
Use of ReSTIR for direct illumination requires generating many independent samples from the full set of lights. If the number of lights is very large, the data for the lights requires a large amount of memory, and random sampling will require slow memory accesses. This has been addressed by pre-generating sets of light samples shared by tiles of adjacent pixels in a single frame.

Noise and correlation

Because ReSTIR is a stochastic technique its output is noisy, and it is typically used with a denoising pass. Spatial reuse can cause unwanted correlations in the samples for neighboring pixels, visible as blotchiness, usually because relatively few "good" samples are found and then reused many times in a neighborhood. Denoisers may have difficultly dealing with these correlations, e.g. may treat them as surface detail, producing artifacts in the output. Some denoisers also expect the noise in consecutive frames to be independent, and cannot handle the temporal correlations introduced by ReSTIR.
In some situations, unbiased ReSTIR can increase variance compared to simpler techniques, for example around shadow edges when using ReSTIR for direct illumination. Partly for this reason biased rendering, which causes darkening around edges but has less visible noise, may be preferred.

Color noise

ReSTIR's streaming RIS technique retains a small number of samples in each reservoir, typically only a single sample, regardless of how many samples have been examined. This single sample is used to determine the final pixel color. If luminance is used as the target function during importance sampling, the rendered pixel value will converge to the correct brightness, however it will have a color determined by a single sample, which introduces color noise noticeable in scenes where light from different directions has different colors, and the pixel color will not converge to the correct value. This problem does not occur with standard Monte Carlo integration because multiple samples are averaged. Correlations between adjacent pixels may make this color noise more noticeable by producing "clumps" where the same sample was used.
For real-time rendering, "decoupling" sample reuse and rendering, so up to three samples are used for determining pixel color, can reduce color noise, but sometimes increases noise because lower quality samples are used in addition to the sample chosen by reservoir merging. For off-line rendering, color noise is usually reduced or avoided by using the average of the samples produced by many repetitions of ReSTIR, instead of using a single sample for rendering.

Relationship to other techniques

Denoising

ReSTIR was developed while exploring ways to add ray tracing to real-time rendering. Due to hardware limitations, only a relatively small number of rays can be traced per pixel per frame, even on high-end GPUs. This means that stochastic ray tracing techniques can only use a small number of samples, producing output that is very noisy, and a denoising pass is essential for acceptable output. One of the goals of the work that led to ReSTIR was to provide better input samples to denoisers, and early ReSTIR demos used denoisers.
Denoisers are implemented using techniques such as bilateral filters, or neural networks. Simpler denoisers work with pixel color values only, while more advanced implementations use additional data as input, including surface normal and information about the specular reflections and surface material. Denoisers are limited to working with information that is in the input values, and samples generated by ReSTIR can provide more information.

Light sampling

Traditional path tracing uses data structures such as light trees to allow importance sampling of lights, favoring lights that are likely to appear brighter at the point in the scene being rendered. The disadvantages of such techniques are that the data structures may be expensive to update if the lights move or change, may be expensive to query in GPU shader code, and generally do not capture shadowing information.
ReSTIR can be used as an alternative to these data structures, or as a way to augment them with shadowing information. Because the RIS portion of ReSTIR permits using lower-quality initial samples ReSTIR may allow using light sampling data structures that are easier to update dynamically when lights move, and cheaper to query, without compromising quality. However, in scenes where there are many lights but few of them are visible at any given point, using a cheap initial sampling method with ReSTIR may not be an effective approach. Better light sampling techniques can be used to generate the initial samples, if the quality increase justifies the cost, but this may be counterproductive in practice.

Path guiding

Path guiding techniques for path tracing use data structures, built during an initial pass or dynamically during rendering, that help make sampling more efficient by preferentially sampling path directions that are likely to contribute more light. These data structures are usually in world space rather than screen space, making them complex to update and query.
Chris Wyman has said that ReSTIR can be viewed as a kind of path guiding, working in screen space. Sharing of statistical information between frames and neighboring pixels allows the distribution of samples in each reservoir to converge to an optimal sampling distribution.
A combination of ReSTIR and path guiding, called ReSTIR-PG, uses samples produced by ReSTIR to determine distributions for path guiding in the following frame.

Metropolis light transport

Like ReSTIR, Metropolis light transport is a rendering algorithm that aims to sample light paths with probability proportional to how much light each path contributes to the image. MLT can render lighting effects such as caustics more efficiently than standard path tracing. It uses Metropolis sampling, a method that allows sampling a probability distribution when a function proportional to the probability density function is known, a property shared with importance resampling, which is the basis of the resampled importance sampling technique used in ReSTIR. The original version of MLT used a procedure similar to RIS to generate initial samples.
Both MLT and the generalized path tracing version of ReSTIR take paths that were previously sampled, modify them, and reuse them stochastically. MLT makes random changes to paths, and paths can be modified in such a way that they now contribute to a different pixel in the image. In contrast, the shift mappings in GRIS are deterministic rather than random, and they take paths from other pixels or other frames and change them so that they connect to a chosen pixel.
A version of ReSTIR for path tracing has been developed that uses Metropolis sampling in addition to spatiotemporal reuse, though it does not use the full MLT algorithm. After performing standard ReSTIR spatiotemporal reuse, paths are randomly mutated, which reduces the correlations between pixels caused by spatial reuse and allows a greater amount of spatial reuse without visible artifacts. Increasing spatial reuse can enable convergence for images with difficult indirect and specular lighting, when otherwise the image would have a "boiling" appearance during real-time rendering due to frequent replacement of samples.

Use in games

The developers of Cyberpunk 2077 adopted ReSTIR DI in order to support rendering night-time city scenes with a large number of lights, including moving lights from vehicles, which was not possible with the shadow maps typically used for real-time rendering. ReSTIR GI is used selectively for indirect lighting on rough surfaces, in combination with a radiance cache.
One of the developers of Tiny Glade, Tomasz Stachowiak has said that they tried ReSTIR for global illumination and it produced high quality results, but they were able to switch to simpler Monte Carlo integration because sampling variance is low for the indirect light in their game. However they were experimenting with using ReSTIR DI to support a large number of direct lights.