AoS and SoA
In computing, an array of structures , structure of arrays or array of structures of arrays are contrasting ways to arrange a sequence of records in memory, with regard to interleaving, and are of interest in SIMD and SIMT programming.
Structure of arrays
Structure of arrays is a layout separating elements of a record into one parallel array per field. The motivation is easier manipulation with packed SIMD instructions in most instruction set architectures, since a single SIMD register can load homogeneous data, possibly transferred by a wide internal datapath. If only a specific part of the record is needed, only those parts need to be iterated over, allowing more data to fit onto a single cache line. The downside is requiring more cache ways when traversing data, and inefficient indexed addressing.For example, to store N points in 3D space using a structure of arrays:
struct Vector3List ;
struct Vector3List points;
float get_point_x
Array of structures
Array of structures is the opposite layout, in which data for different fields is interleaved.This is often more intuitive, and supported directly by most programming languages.
For example, to store N points in 3D space using an array of structures:
struct Vector3 ;
struct Vector3 points;
float get_point_x
Array of structures of arrays
Array of structures of arrays or tiled array of structs is a hybrid approach between the previous layouts, in which data for different fields is interleaved using tiles or blocks with size equal to the SIMD vector size. This is often less intuitive, but can achieve the memory throughput of the SoA approach, while being more friendly to the cache locality and load port architectures of modern processors. In particular, memory requests in modern processors have to be fulfilled in fixed width. The tiled storage of AoSoA aligns the memory access pattern to the requests' fixed width, leading to fewer access operations to complete a memory request and thus increasing the efficiency.For example, to store N points in 3D space using an array of structures of arrays with a SIMD register width of 8 floats :
struct Vector3x8 ;
struct Vector3x8 points;
float get_point_x
A different width may be needed depending on the actual SIMD register width. The interior arrays may be replaced with SIMD types such as for languages with such support.
Alternatives
4D vectors
AoS vs. SoA presents a choice when considering 3D or 4D vector data on machines with four-lane SIMD hardware. SIMD ISAs are usually designed for homogeneous data, however some provide a dot product instruction and additional permutes, making the AoS case easier to handle.Although most GPU hardware has moved away from 4D instructions to scalar SIMT pipelines, modern compute kernels using SoA instead of AoS can still give better performance due to memory coalescing.
Software support
Most languages support the AoS format more naturally by combining records and various array abstract data types.SoA is mostly found in languages, libraries, or metaprogramming tools used to support a data-oriented design. Examples include:
- "Data frames", as implemented in R, Python's Pandas package, and Julia's DataFrames.jl package, are interfaces to access SoA like AoS.
- The Julia package StructArrays.jl allows for accessing SoA as AoS to combine the performance of SoA with the intuitiveness of AoS.
- Code generators for the C language, including and the X Macro technique.