Pixel-art scaling algorithms
Pixel art scaling algorithms are graphical filters that attempt to enhance the appearance of hand-drawn 2D pixel art graphics. These algorithms are a form of automatic image enhancement. Pixel art scaling algorithms employ methods significantly different than the common methods of image rescaling, which have the goal of preserving the appearance of images.
As pixel art graphics are commonly used at very low resolutions, they employ careful coloring of individual pixels. This results in graphics that rely on a high amount of stylized visual cues to define complex shapes. Several specialized algorithms have been developed to handle re-scaling of such graphics.
These specialized algorithms can improve the appearance of pixel-art graphics, but in doing so they introduce changes. Such changes may be undesirable, especially if the goal is to faithfully reproduce the original appearance.
Since a typical application of this technology is improving the appearance of fourth-generation and earlier video games on arcade and console emulators, many pixel art scaling algorithms are designed to run in real-time for sufficiently small input images at 60-frames per second. This places constraints on the type of programming techniques that can be used for this sort of real-time processing. Many work only on specific scale factors. 2× is the most common scale factor, while and 3×, 4×, 5×, and 6× exist but are less used.
Algorithms
SAA5050 'Diagonal Smoothing'
The Mullard SAA5050 Teletext character generator chip used a primitive pixel scaling algorithm to generate higher-resolution characters on the screen from a lower-resolution representation from its internal ROM. Internally, each character shape was defined on a pixel grid, which was then interpolated by smoothing diagonals to give a pixel character, with a characteristically angular shape, surrounded to the top and the left by two pixels of blank space. The algorithm only works on monochrome source data, and assumes the source pixels will be logically true or false depending on whether they are 'on' or 'off'. Pixels 'outside the grid pattern' are assumed to be off.The algorithm works as follows:
A B C --\ 1 2
D E F --/ 3 4
1 = B |
2 = B |
3 = E |
4 = E |
Note that this algorithm, like the Eagle algorithm below, has a flaw:
If a pattern of 4 pixels in a hollow diamond shape appears, the hollow will be obliterated by the expansion.
The SAA5050's internal character ROM carefully avoids ever using this pattern.
The degenerate case:
*
* *
*
becomes:
**
****
******
******
****
**
EPX/Scale2×/AdvMAME2×
Eric's Pixel Expansion is an algorithm developed by Eric Johnston at LucasArts around 1992, when porting the SCUMM engine games from the IBM PC to the early color Macintosh computers, which ran at more or less double that resolution.The algorithm works as follows, expanding P into 4 new pixels based on P's surroundings:
1=P; 2=P; 3=P; 4=P;
IF CA => 1=A
IF AB => 2=B
IF DC => 3=C
IF BD => 4=D
IF of A, B, C, D, three or more are identical: 1=2=3=4=P
Later implementations of this same algorithm are slightly more efficient but functionally identical:
1=P; 2=P; 3=P; 4=P;
IF CA AND C!=D AND A!=B => 1=A
IF AB AND A!=C AND B!=D => 2=B
IF DC AND D!=B AND C!=A => 3=C
IF BD AND B!=A AND D!=C => 4=D
AdvMAME2× is available in DOSBox via the
scaler=advmame2x dosbox.conf option.The AdvMAME4×/Scale4× algorithm is just EPX applied twice to get 4× resolution.
Scale3×/AdvMAME3× and ScaleFX
The AdvMAME3×/Scale3× algorithm can be thought of as a generalization of EPX to the 3× case. The corner pixels are calculated identically to EPX.
1=E; 2=E; 3=E; 4=E; 5=E; 6=E; 7=E; 8=E; 9=E;
IF DB AND D!=H AND B!=F => 1=D
IF OR => 2=B
IF BF AND B!=D AND F!=H => 3=F
IF OR => 4=D
5=E
IF OR => 6=F
IF HD AND H!=F AND D!=B => 7=D
IF OR => 8=H
IF FH AND F!=B AND H!=D => 9=F
There is also a variant improved over Scale3× called ScaleFX, developed by Sp00kyFox, and a version combined with Reverse-AA called ScaleFX-Hybrid.
Eagle
Assume an input matrix of pixels where the centermost pixel is the pixel to be scaled, and an output matrix of pixels
first: |Then
... --\ CC |S T U --\ 1 2
. C. --/ CC |V C W --/ 3 4
... |X Y Z
| IF VST => 1=S
| IF TUW => 2=U
| IF VXY => 3=X
| IF WZY => 4=Z
Thus if we have a single black pixel on a white background it will vanish. This is a bug in the Eagle algorithm but is solved by other algorithms such as EPX, 2xSaI, and HQ2x.
2×SaI
2×SaI, short for 2× Scale and Interpolation engine, was inspired by Eagle. It was designed by Derek Liauw Kie Fa, also known as Kreed, primarily for use in console and computer emulators, and it has remained fairly popular in this niche. Many of the most popular emulators, including ZSNES and VisualBoyAdvance, offer this scaling algorithm as a feature. Several slightly different versions of the scaling algorithm are available, and these are often referred to as Super 2×SaI and Super Eagle.The 2xSaI family works on a matrix of pixels where the pixel marked A below is scaled:
I E F J
G A B K --\ W X
H C D L --/ Y Z
M N O P
For 16-bit pixels, they use pixel masks which change based on whether the 16-bit pixel format is 565 or 555. The constants
colorMask, lowPixelMask, qColorMask, qLowPixelMask, redBlueMask, and greenMask are 16-bit masks. The lower 8 bits are identical in either pixel format.Two interpolation functions are described:
INTERPOLATE.
-- linear midpoint of A and B
if return A;
return
+
+ ;
Q_INTERPOLATE
-- bilinear interpolation; A, B, C, and D's average
x =
+
+
+ ;
y =
+
+
+ ;
y = & qLowPixelMask;
return x + y;
The algorithm checks A, B, C, and D for a diagonal match such that
AD and B!=C, or the other way around, or if they are both diagonals or if there is no diagonal match. Within these, it checks for three or four identical pixels. Based on these conditions, the algorithm decides whether to use one of A, B, C, or D, or an interpolation among only these four, for each output pixel. The 2xSaI arbitrary scaler can enlarge any image to any resolution and uses bilinear filtering to interpolate pixels.Since Kreed released the source code under the GNU General Public License, it is freely available to anyone wishing to utilize it in a project released under that license. Developers wishing to use it in a non-GPL project would be required to rewrite the algorithm without using any of Kreed's existing code.
It is available in DosBox via
scaler=2xsai option.hq''n''x family
Maxim Stepin's hq2x, hq3x, and hq4x are for scale factors of 2:1, 3:1, and 4:1 respectively. Each work by comparing the color value of each pixel to those of its eight immediate neighbors, marking the neighbors as close or distant, and using a pre-generated lookup table to find the proper proportion of input pixels' values for each of the 4, 9 or 16 corresponding output pixels. The hq3x family will perfectly smooth any diagonal line whose slope is ±0.5, ±1, or ±2 and which is not anti-aliased in the input; one with any other slope will alternate between two slopes in the output. It will also smooth very tight curves. Unlike 2xSaI, it anti-aliases the output.hqnx was initially created for the Super NES emulator ZSNES. The author of bsnes has released a space-efficient implementation of hq2x to the public domain. A port to shaders, which has comparable quality to the early versions of xBR, is available. Before the port, a shader called "scalehq" has often been confused for hqx.
xBR family
There are 6 filters in this family: xBR , xBRZ, xBR-Hybrid, Super xBR, xBR+3D and Super xBR+3D.xBR, created by Hyllian, works much the same way as HQx and would generate the same result as HQx when given the above pattern. However, it goes further than HQx by using a 2-stage set of interpolation rules, which better handle more complex patterns such as anti-aliased lines and curves. Scaled background textures keep the sharp characteristics of the original image, rather than becoming blurred like HQx tends to do. The newest xBR versions are multi-pass and can preserve small details better. There is also a version of xBR combined with Reverse-AA shader called xBR-Hybrid. xBR+3D is a version with a 3D mask that only filters 2D elements.
xBRZ by Zenju is a modified version of xBR. It is implemented from scratch as a CPU-based filter in C++. It uses the same basic idea as xBR's pattern recognition and interpolation but with a different rule set designed to preserve fine image details as small as a few pixels. This makes it useful for scaling the details in faces, and in particular eyes. xBRZ is optimized for multi-core CPUs and 64-bit architectures and shows 40–60% better performance than HQx even when running on a single CPU core only. It supports scaling images with an alpha channel, and scaling by integer factors from 2× up to 6×.
Super xBR is an algorithm developed by Hylian in 2015. It uses some combinations of known linear filters along with xBR edge detection rules in a non-linear way. It works in two passes and can only scale an image by two. Super xBR+3D is a version with a 3D mask that only filters 2D elements.
There is also a Super xBR version rewritten in C/C++.
RotSprite
RotSprite is a scaling and rotation algorithm for sprites developed by Xenowhirl. It produces far fewer artifacts than nearest-neighbor rotation algorithms, and like EPX, it does not introduce new colors into the image.The algorithm first scales the image to 8 times its original size with a modified Scale2× algorithm which treats similar pixels as matches. It then calculates what rotation offset to use by favoring sampled points that are not boundary pixels. Next, the rotated image is created with a nearest-neighbor scaling and rotation algorithm that simultaneously shrinks the big image back to its original size and rotates the image. Finally, overlooked single-pixel details are restored if the corresponding pixel in the source image is different and the destination pixel has three identical neighbors.