Chroma key

Chroma key compositing, or chroma keying, is a visual-effects and post-production technique for compositing two or more images or video streams together based on colour hues. The technique has been used in many fields to remove a background from the subject of a photo or video — particularly the newscasting, motion picture, and video game industries. A colour range in the foreground footage is made transparent, allowing separately filmed background footage or a static image to be inserted into the scene. The chroma keying technique is commonly used in video production and post-production. This technique is also referred to as colour keying, colour separation overlay, or by various terms for specific colour-related variants such as green screen or blue screen; chroma keying can be done with backgrounds of any colour that are uniform and distinct, but green and blue backgrounds are more commonly used because they differ most distinctly in hue from any human skin colour. No part of the subject being filmed or photographed may duplicate the colour used as the backing, or the part may be erroneously identified as part of the backing.
It is commonly used for live weather forecast broadcasts in which a news presenter is seen standing in front of a CGI map instead of a large blue or green background. Chroma keying is also common in the entertainment industry for visual effects in filmmaking and video games. Rotoscopy may instead be carried out on subjects that are not in front of a green screen. Motion tracking can also be used in conjunction with chroma keying, such as to move the background as the subject moves.

History

Predecessors

Prior to the introduction of travelling mattes and optical printing, double exposure was used to introduce elements into a scene which were not present in the initial exposure. This was done using black draping where a green screen would be used today. George Albert Smith first used this approach in 1898. In 1903, The Great Train Robbery by Edwin S. Porter used double exposure to add background scenes to windows which were black when filmed on set, using a garbage matte to expose only the window areas.
In order to have figures in one exposure actually move in front of a substituted background in the other, a travelling matte was needed, to occlude the correct portion of the background in each frame. In 1918, Frank Williams patented a travelling matte technique, again based on using a black background. This was used in many films, such as The Invisible Man.
In the 1920s, Walt Disney used a white backdrop to include human actors with cartoon characters and backgrounds in his Alice Comedies.

From bluescreen to greenscreen

The blue screen method was developed in the 1930s at RKO Radio Pictures. At RKO, Linwood Dunn used an early version of the travelling matte to create "wipes" — where there were transitions like a windshield wiper in films such as Flying Down to Rio. Credited to Larry Butler, a scene featuring a genie escaping from a bottle was the first use of a proper bluescreen process to create a travelling matte for The Thief of Bagdad, which won the Academy Award for Best Special Effects that year. In 1950, Warner Brothers employee and ex-Kodak researcher Arthur Widmer began working on an ultraviolet travelling matte process. He also began developing bluescreen techniques: one of the first films to use them was the 1958 adaptation of the Ernest Hemingway novella, The Old Man and the Sea, starring Spencer Tracy.
The name "Chroma-Key" was RCA's trade name for the process, as used on its NBC television broadcasts, incorporating patents granted to RCA's Albert N. Goldsmith. A very early broadcast use was NBC's George Gobel Show in fall 1957.
Petro Vlahos was awarded an Academy Award for his refinement of these techniques in 1964. His technique exploits the fact that most objects in real-world scenes have a colour whose blue-colour component is similar in intensity to their green-colour component. Zbigniew Rybczyński also contributed to bluescreen technology. An optical printer with two projectors, a film camera and a "beam splitter", was used to combine the actor in front of a blue screen together with the background footage, one frame at a time. In the early 1970s, American and British television networks began using green backdrops instead of blue for their newscasts. During the 1980s, minicomputers were used to control the optical printer. For the film The Empire Strikes Back, Richard Edlund created a "quad optical printer" that accelerated the process considerably and saved money. He received a special Academy Award for his innovation.
For decades, travelling matte shots had to be done "locked-down", so that neither the matted subject nor the background could shift their camera perspective at all. Later, computer-timed, motion-control cameras alleviated this problem, as both the foreground and background could be filmed with the same camera moves.
Meteorologists on television often use a field monitor, to the side of the screen, to see where they are putting their hands against the background images. A newer technique is to project a faint image onto the screen.
Some films make heavy use of chroma key to add backgrounds that are constructed entirely using computer-generated imagery. Performances from different takes can be composited together, which allows actors to be filmed separately and then placed together in the same scene. Chroma key allows performers to appear to be in any location without leaving the studio.
Advances in computer technology have simplified the incorporation of motion into composited shots, even when using handheld cameras. Reference points such as a painted grid, X's marked with tape, or equally spaced tennis balls attached to the wall, can be placed onto the coloured background to serve as markers. In post-production, a computer can use these markers to compute the camera's position and thus render an image that matches the perspective and movement of the foreground perfectly. Modern advances in software and computational power have eliminated the need to accurately place the markers ⁠ — ⁠the software figures out their position in space; a potential disadvantage of this is that it requires camera movement, possibly contributing to modern cinematographic techniques whereby the camera is always in motion.

Process

The principal subject is filmed or photographed against a background consisting of a single colour or a relatively narrow range of colours, usually blue or green because these colours are considered to be the furthest away from skin tone. The portions of the video which match the pre-selected colour are replaced by the alternate background video. This process is commonly known as "keying", "keying out" or simply a "key".

Processing a green backdrop

Green is used as a backdrop for TV and electronic cinematography more than any other colour because television weather presenters tended to wear blue suits. When chroma keying first came into use in television production, the blue screen that was then the norm in the movie industry was used out of habit, until other practical considerations caused the television industry to move from blue to green screens. Broadcast-quality colour television cameras use separate red, green, and blue image sensors, and early analog TV chroma keyers required RGB component video to work reliably. From a technological perspective it was equally possible to use the blue or green channel, but because blue clothing was an ongoing challenge, the green screen came into common use. Newscasters sometimes forget the chroma key dress code, and when the key is applied to clothing of the same colour as the background, the person would seem to disappear into the key. Because green clothing is less common than blue, it soon became apparent that it was easier to use a green matte screen than it was to constantly police the clothing choices of on-air talent. Also, because the human eye is more sensitive to green wavelengths, which lie in the middle of the visible light spectrum, the green analog video channel typically carried more signal strength, giving a better signal to noise ratio compared to the other component video channels, so green screen keys could produce the cleanest key. In the digital television and cinema age, much of the tweaking that was required to make a good quality key has been automated. However, the one constant that remains is some level of colour coordination to keep foreground subjects from being keyed out.

Processing a blue backdrop

Before electronic chroma keying, compositing was done on film. The camera colour negative was printed onto high-contrast black and white negative, using either a filter or the high contrast film's colour sensitivity to expose only blue frequencies. Blue light only shines through the colour negative where there is not blue in the scene, so this left the film clear where the blue screen was, and opaque elsewhere, except it also produced clear for any white objects. Removing these spots could be done by a suitable double-exposure with the colour positive, and many other techniques. The result was film that was clear where the blue screen was, and opaque everywhere else. This is called a female matte, similar to an alpha matte in digital keying. Copying this film onto another high-contrast negative produced the opposite male matte. The background negative was then packed with the female matte and exposed onto a final strip of film, then the camera negative was packed with the male matte and was double-printed onto this same film. These two images combined creates the final effect.

Major factors

The most important factor for a key is the colour separation of the foreground and background – a blue screen will be used if the subject is predominantly green, despite the camera being more sensitive to green light.
In analog television, colour is represented by the phase of the chroma subcarrier relative to a reference oscillator. Chroma key is achieved by comparing the phase of the video to the phase corresponding to the pre-selected colour. In-phase portions of the video are replaced by the alternate background video.
In digital colour TV, colour is represented by three numbers. Chroma key is achieved by a simple numerical comparison between the video and the pre-selected colour. If the colour at a particular point on the screen matches, then the video at that point is replaced by the alternate background.