Deinterlacing


Deinterlacing is the process of converting interlaced video into a non-interlaced or progressive form. Interlaced video signals are commonly found in analog television, VHS, Betamax, Video 8, digital home video tapes such as Digital8 and DV tapes, CED Disc, Laserdisc, digital television when in the 1080i format, some DVD titles, and a smaller number of Blu-ray discs.
An interlaced video frame consists of two fields taken in sequence: the first containing all the odd lines of the image, and the second all the even lines. Analog television employed this technique because it allowed for less transmission bandwidth while keeping a high frame rate for smoother and more life-like motion. A non-interlaced signal that uses the same bandwidth only updates the display half as often and was found to create a perceived flicker or stutter. CRT-based displays were able to display interlaced video correctly due to their complete analog nature, blending in the alternating lines seamlessly. However, since the early 2000s, displays such as televisions and computer monitors have become almost entirely digital—in that the display is composed of discrete pixels—and on such displays the interlacing becomes noticeable and can appear as a distracting visual defect. The deinterlacing process should try to minimize these.
Deinterlacing is thus a necessary process and comes built-in to most modern DVD players, Blu-ray players, LCD/LED televisions, digital projectors, TV set-top boxes, professional broadcast equipment, and computer video players and editors—although each with varying levels of quality.
Deinterlacing has been researched for decades and employs complex processing algorithms; however, consistent results have been very hard to achieve.

Background

Both video and photographic film capture a series of frames in rapid succession; however, television systems read the captured image by serially scanning the image sensor by lines. In analog television, each frame is divided into two consecutive fields, one containing all even lines, another with the odd lines. The fields are captured in succession at a rate twice that of the nominal frame rate. For instance, PAL and SECAM systems have a rate of 25 frames/sec or 50 fields/sec, while the NTSC system delivers 29.97 frames/sec or 59.94 fields/sec. This process of dividing frames into half-resolution fields at double the frame rate is known as interlacing.
Since the interlaced signal contains the two fields of a video frame shot at two different times, it enhances motion perception to the viewer and reduces flicker by taking advantage of the persistence of vision effect. This results in an effective doubling of time resolution as compared with non-interlaced footage. However, interlaced signal requires a display that is natively capable of showing the individual fields in a sequential order, and only traditional CRT-based TV sets are capable of displaying interlaced signal, due to the electronic scanning and lack of apparent fixed resolution.
Most modern displays, such as LCD, DLP and plasma displays, are not able to work in interlaced mode, because they are fixed-resolution displays and only support progressive scanning. In order to display interlaced signal on such displays, the two interlaced fields must be converted to one progressive frame with a process known as de-interlacing. However, when the two fields taken at different points in time are re-combined to a full frame displayed at once, visual defects called interlace artifacts or combing occur with moving objects in the image. A good deinterlacing algorithm should try to avoid interlacing artifacts as much as possible and not sacrifice image quality in the process, which is hard to achieve consistently. There are several techniques available that extrapolate the missing picture information, however they rather fall into the category of intelligent frame creation and require complex algorithms and substantial processing power.
Deinterlacing techniques require complex processing and thus can introduce a delay into the video feed. While not generally noticeable, this can result in the display of older video games lagging behind controller input. Many TVs thus have a "game mode" in which minimal processing is done in order to maximize speed at the expense of image quality. Deinterlacing is only partly responsible for such lag; scaling also involves complex algorithms that take milliseconds to run.

Progressive source material

Some interlaced video may have been originally created from progressive footage, and the deinterlacing process should consider this as well.
Typical movie material is shot on 24 frames/s film. Converting film to interlaced video typically uses a process called telecine whereby each frame is converted to multiple fields. In some cases, each film frame can be presented by exactly two progressive segmented frames, and in this format it does not require a complex deinterlacing algorithm because each field contains a part of the very same progressive frame. However, to match 50 field interlaced PAL/SECAM or 59.94/60 field interlaced NTSC signal, frame rate conversion is necessary using various "pulldown" techniques. Most advanced TV sets can restore the original 24 frame/s signal using an inverse telecine process. Another option is to speed up 24-frame film by 4% for PAL/SECAM conversion; this method is still widely used for DVDs, as well as television broadcasts in the PAL markets.
DVDs can either encode movies using one of these methods, or store original 24 frame/s progressive video and use MPEG-2 decoder tags to instruct the video player on how to convert them to the interlaced format. Most movies on Blu-rays have preserved the original non interlaced 24 frame/s motion film rate and allow output in the progressive 1080p24 format directly to display devices, with no conversion necessary.
Some 1080i HDV camcorders also offer PsF mode with cinema-like frame rates of 24 or 25 frame/s. TV production crews can also use special film cameras which operate at 25 or 30 frame/s, where such material does not need framerate conversion for broadcasting in the intended video system format.

Deinterlacing methods

Deinterlacing requires the display to buffer one or more fields and recombine them into full frames. In theory this would be as simple as capturing one field and combining it with the next field to be received, producing a single frame. However, the originally recorded signal was produced from two fields at different points in time, and without special processing any motion across the fields usually results in a "combing" effect where alternate lines are slightly displaced from each other.
There are various methods to deinterlace video, each producing different problems or artifacts of its own. Some methods are much cleaner in artifacts than other methods.
Most deinterlacing techniques fall under three broad groups:
  1. [|Field combination deinterlacing] which takes the even and odd fields and combine them into one frame. This halves the perceived frame-rate whereby 50i or 60i is converted to 25p or 30p.
  2. [|Field extension deinterlacing] which takes each field and extend it to the entire screen to make a frame. This halves the vertical resolution of the image but maintains the original field-rate.
  3. [|Motion compensation deinterlacing] which uses more advanced algorithms to detect motion across fields, switching techniques when necessary. This produces the best quality result, but requires the most processing power.
Modern deinterlacing systems therefore buffer several fields and use techniques like edge detection in an attempt to find the motion between the fields. This is then used to interpolate the missing lines from the original field, reducing the combing effect.

Field combination deinterlacing

These methods take the even and odd fields and combine them into one frame. They retain the full vertical resolution at the expense of the temporal resolution whereby 50i/60i is converted to 24p/25p/30p which may lose the smooth, fluid feel of the original. However, if the interlaced signal was originally produced from a lower frame-rate source such as film, then no information is lost and these methods may suffice.
  • Weaving is the simplest and most rudimentary method, performed by interleaving the consecutive fields together into a single frame. This method does not cause any problems when the image has not changed between fields, but any motion will result in artifacts known as "combing" when the pixels in one field do not line up with the pixels in the other, forming a jagged edge.
  • Blending is done by blending, or averaging consecutive fields to be displayed as one frame. Combing is avoided because the images are on top of each other. This instead leaves an artifact known as ghosting. The image loses both vertical resolution and temporal resolution. Although video produced with this technique only requires half the number of pixels vertically, it is often combined with a vertical resize so that the output has no numerical loss in vertical pixels. When interpolation is used, it can result in an even softer image. Blending also loses half the temporal resolution since two motion fields are combined into one frame.
  • Selective blending, or smart blending or motion adaptive blending, is a combination of weaving and blending. As areas that haven't changed from frame to frame don't need any processing, the frames are woven and only the areas that need it are blended. This retains the full vertical resolution and half the temporal resolution, and it has fewer artifacts than weaving or blending because of the selective combination of both techniques.
  • Inverse Telecine: Telecine is used to convert a motion picture source at 24 frames per second to interlaced TV video in countries that use NTSC video system at 30 frames per second. Countries which use PAL at 25 frames per second do not require Telecine – motion picture sources are merely sped up 4% to achieve the needed 25 frames per second. If Telecine was used then it is possible to reverse the algorithm to obtain the original non-interlaced footage, which has a slower frame rate. In order for this to work, the exact telecine pattern must be known or guessed. Unlike most other deinterlacing methods, when it works, inverse telecine can perfectly recover the original progressive video stream.
  • Telecine-style algorithms: If the interlaced footage was generated from progressive frames at a slower frame rate, then the exact original frames can be recovered by copying the missing field from a matching previous/next frame. In cases where there is no match, then the filter falls back on another deinterlacing method such as blending or line-doubling. This means that the worst case for Telecine is occasional frames with ghosting or reduced resolution. By contrast, when more sophisticated motion-detection algorithms fail, they can introduce pixel artifacts that are unfaithful to the original material. For telecine video, decimation can be applied as a post-process to reduce the frame rate, and this combination is generally more robust than a simple inverse telecine, which fails when differently interlaced footage is spliced together.