Advanced Vector Extensions


Advanced Vector Extensions are SIMD extensions to the x86 instruction set architecture for microprocessors from Intel and Advanced Micro Devices. They were proposed by Intel in March 2008 and first supported by Intel with the Sandy Bridge microarchitecture shipping in Q1 2011 and later by AMD with the Bulldozer microarchitecture shipping in Q4 2011. AVX provides new features, new instructions, and a new coding scheme.
AVX2 expands most integer commands to 256 bits and introduces new instructions. They were first supported by Intel with the Haswell microarchitecture, which shipped in 2013.
AVX-512 expands AVX to 512-bit support using a new EVEX prefix encoding proposed by Intel in July 2013 and first supported by Intel with the Knights Landing co-processor, which shipped in 2016. In conventional processors, AVX-512 was introduced with Skylake server and HEDT processors in 2017.

Advanced Vector Extensions

AVX uses sixteen YMM registers to perform a single instruction on multiple pieces of data. Each YMM register can hold and do simultaneous operations on:
The width of the SIMD registers is increased from 128 bits to 256 bits, and renamed from XMM0–XMM7 to YMM0–YMM7. The legacy SSE instructions can still be utilized via the VEX prefix to operate on the lower 128 bits of the YMM registers.
511 256255 128127 0
ZMM0 YMM0 XMM0
ZMM1YMM1XMM1
ZMM2YMM2XMM2
ZMM3YMM3XMM3
ZMM4YMM4XMM4
ZMM5YMM5XMM5
ZMM6YMM6XMM6
ZMM7YMM7XMM7
ZMM8YMM8XMM8
ZMM9YMM9XMM9
ZMM10YMM10XMM10
ZMM11YMM11XMM11
ZMM12YMM12XMM12
ZMM13YMM13XMM13
ZMM14YMM14XMM14
ZMM15YMM15XMM15
ZMM16YMM16XMM16
ZMM17YMM17XMM17
ZMM18YMM18XMM18
ZMM19YMM19XMM19
ZMM20YMM20XMM20
ZMM21YMM21XMM21
ZMM22YMM22XMM22
ZMM23YMM23XMM23
ZMM24YMM24XMM24
ZMM25YMM25XMM25
ZMM26YMM26XMM26
ZMM27YMM27XMM27
ZMM28YMM28XMM28
ZMM29YMM29XMM29
ZMM30YMM30XMM30
ZMM31YMM31XMM31

AVX introduces a three-operand SIMD instruction format called VEX coding scheme, where the destination register is distinct from the two source operands. For example, an SSE instruction using the conventional two-operand form can now use a non-destructive three-operand form, preserving both source operands. Originally, AVX's three-operand format was limited to the instructions with SIMD operands, and did not include instructions with general purpose registers. It was later used for coding new instructions on general purpose registers in later extensions, such as BMI. VEX coding is also used for instructions operating on the k0-k7 mask registers that were introduced with AVX-512.
The alignment requirement of SIMD memory operands is relaxed. Unlike their non-VEX coded counterparts, most VEX coded vector instructions no longer require their memory operands to be aligned to the vector size. Notably, the VMOVDQA instruction still requires its memory operand to be aligned.
The new VEX coding scheme introduces a new set of code prefixes that extends the opcode space, allows instructions to have more than two operands, and allows SIMD vector registers to be longer than 128 bits. The VEX prefix can also be used on the legacy SSE instructions giving them a three-operand form, and making them interact more efficiently with AVX instructions without the need for VZEROUPPER and VZEROALL.
The AVX instructions support both 128-bit and 256-bit SIMD. The 128-bit versions can be useful to improve old code without needing to widen the vectorization, and avoid the penalty of going from SSE to AVX; they are also faster on some early AMD implementations of AVX. This mode is sometimes known as AVX-128.
Compared to SSE series, AVX further enhanced performance for digital media playback, web browsing, asymmetric encryption, and others.

New instructions

These AVX instructions are in addition to the ones that are 256-bit extensions of the legacy 128-bit SSE instructions; most are usable on both 128-bit and 256-bit operands.
InstructionDescription
VBROADCASTSS, VBROADCASTSD, VBROADCASTF128Copy a 32-bit, 64-bit or 128-bit memory operand to all elements of a XMM or YMM vector register.
VINSERTF128Replaces either the lower half or the upper half of a 256-bit YMM register with the value of a 128-bit source operand. The other half of the destination is unchanged.
VEXTRACTF128Extracts either the lower half or the upper half of a 256-bit YMM register and copies the value to a 128-bit destination operand.
VMASKMOVPS, VMASKMOVPDConditionally reads any number of elements from a SIMD vector memory operand into a destination register, leaving the remaining vector elements unread and setting the corresponding elements in the destination register to zero. Alternatively, conditionally writes any number of elements from a SIMD vector register operand to a vector memory operand, leaving the remaining elements of the memory operand unchanged. On the AMD Jaguar processor architecture, this instruction with a memory source operand takes more than 300 clock cycles when the mask is zero, in which case the instruction should do nothing. This appears to be a design flaw.
VPERMILPS, VPERMILPDPermute In-Lane. Shuffle the 32-bit or 64-bit vector elements of one input operand. These are in-lane 256-bit instructions, meaning that they operate on all 256 bits with two separate 128-bit shuffles, so they can not shuffle across the 128-bit lanes.
VPERM2F128Shuffle the four 128-bit vector elements of two 256-bit source operands into a 256-bit destination operand, with an immediate constant as selector.
VTESTPS, VTESTPDPacked bit test of the packed single-precision or double-precision floating-point sign bits, setting or clearing the ZF flag based on AND and CF flag based on ANDN.
VZEROALLSet all YMM registers to zero and tag them as unused. Used when switching between 128-bit use and 256-bit use.
VZEROUPPERSet the upper half of all YMM registers to zero. Used when switching between 128-bit use and 256-bit use.

CPUs with AVX

Issues regarding compatibility between future Intel and AMD processors are discussed under XOP instruction set.
  • VIA:
  • * Nano QuadCore
  • * Eden X4
  • Zhaoxin:
  • * WuDaoKou-based processors

Compiler and assembler support

  • Absoft supports with - flag.
  • The Free Pascal compiler supports AVX and AVX2 with the -CfAVX and -CfAVX2 switches from version 2.7.1.
  • RAD studio supports AVX2 and AVX512.
  • The GNU Assembler inline assembly functions support these instructions, as do Intel primitives and the Intel inline assembler. GAS supports AVX starting with binutils version 2.19.
  • GCC starting with version 4.6 and the Intel Compiler Suite starting with version 11.1 support AVX.
  • The Open64 compiler version 4.5.1 supports AVX with - flag.
  • PathScale supports via the - flag.
  • The Vector Pascal compiler supports AVX via the - flag.
  • The Visual Studio 2010/2012 compiler supports AVX via intrinsic and switch.
  • NASM starting with version 2.03 and newer. There were numerous bug fixes and updates related to AVX in version 2.04.
  • Other assemblers such as MASM VS2010 version, YASM, FASM and JWASM.

Operating system support

AVX adds new register-state through the 256-bit wide YMM register file, so explicit operating system support is required to properly save and restore AVX's expanded registers between context switches. The following operating system versions support AVX:

Advanced Vector Extensions 2

Advanced Vector Extensions 2, also known as Haswell New Instructions, is an expansion of the AVX instruction set introduced in Intel's Haswell microarchitecture. AVX2 makes the following additions:
  • expansion of most vector integer SSE and AVX instructions to 256 bits
  • Gather support, enabling vector elements to be loaded from non-contiguous memory locations
  • DWORD- and QWORD-granularity any-to-any permutes
  • vector shifts.
Sometimes three-operand fused multiply-accumulate extension is considered part of AVX2, as it was introduced by Intel in the same processor microarchitecture. This is a separate extension using its own CPUID flag, described on its own page and not below.

CPUs with AVX2

  • Intel
  • * Haswell processors and newer, except models branded as Celeron and Pentium.
  • * Celeron and Pentium branded processors starting with Tiger Lake and newer.
  • AMD
  • * Excavator processors and newer.
  • VIA:
  • * Nano QuadCore
  • * Eden X4

AVX-512

AVX-512 are 512-bit extensions to the 256-bit Advanced Vector Extensions SIMD instructions for x86 instruction set architecture proposed by Intel in July 2013.
AVX-512 instructions are encoded with the new EVEX prefix. It allows 4 operands, 8 new 64-bit opmask registers, scalar memory mode with automatic broadcast, explicit rounding control, and compressed displacement memory addressing mode. The width of the register file is increased to 512 bits and total register count increased to 32 in x86-64 mode.
AVX-512 consists of multiple instruction subsets, not all of which are meant to be supported by all processors implementing them. The instruction set consists of the following:
  • AVX-512 Foundation – adds several new instructions and expands most 32- and 64-bit floating-point SSE-SSE4.1 and AVX/AVX2 instructions with EVEX coding scheme to support the 512-bit registers, operation masks, parameter broadcasting, and embedded rounding and exception control
  • AVX-512 Conflict Detection Instructions – efficient conflict detection to allow more loops to be vectorized, supported by Knights Landing
  • AVX-512 Exponential and Reciprocal Instructions – exponential and reciprocal operations designed to help implement transcendental operations, supported by Knights Landing
  • AVX-512 Prefetch Instructions – new prefetch capabilities, supported by Knights Landing
  • AVX-512 Vector Length Extensions – extends most AVX-512 operations to also operate on XMM and YMM registers
  • AVX-512 Byte and Word Instructions – extends AVX-512 to cover 8-bit and 16-bit integer operations
  • AVX-512 Doubleword and Quadword Instructions – enhanced 32-bit and 64-bit integer operations
  • AVX-512 Integer Fused Multiply Add – fused multiply add for 512-bit integers.
  • AVX-512 Vector Byte Manipulation Instructions adds vector byte permutation instructions which are not present in AVX-512BW.
  • AVX-512 Vector Neural Network Instructions Word variable precision – vector instructions for deep learning.
  • AVX-512 Fused Multiply Accumulation Packed Single precision – vector instructions for deep learning.
  • VPOPCNTDQ – count of bits set to 1.
  • VPCLMULQDQ – carry-less multiplication of quadwords.
  • AVX-512 Vector Neural Network Instructions – vector instructions for deep learning.
  • AVX-512 Galois Field New Instructions – vector instructions for calculating Galois field.
  • AVX-512 Vector AES instructions – vector instructions for AES coding.
  • AVX-512 Vector Byte Manipulation Instructions 2 – byte/word load, store and concatenation with shift.
  • AVX-512 Bit Algorithms – byte/word bit manipulation instructions expanding VPOPCNTDQ.
  • AVX-512 Bfloat16 Floating-Point Instructions – vector instructions for AI acceleration.
  • AVX-512 Half-Precision Floating-Point Instructions – vector instructions for operating on floating-point and complex numbers with reduced precision.
  • AVX-512 Bit Manipulation Instructions – bit matrix multiply and bit reversal instructions.
Only the core extension AVX-512F is required by all implementations, though all current implementations also support CD. All central processors with AVX-512 also support VL, DQ and BW. The ER, PF, 4VNNIW and 4FMAPS instruction set extensions are currently only implemented in Intel computing coprocessors.
The updated SSE/AVX instructions in AVX-512F use the same mnemonics as AVX versions; they can operate on 512-bit ZMM registers, and will also support 128/256 bit XMM/YMM registers and byte, word, doubleword and quadword integer operands.
Discontinued subsets include:
  • AVX-512 Vector Pair Intersection to a Pair of Mask Registers – Compute intersection between doublewords/quadwords to a pair of mask registers. Discontinued by Intel, but still supported by AMD.
  • Xeon Phi ER, PF, 4FMAPS, 4VNNIW.

AVX-512 CPU compatibility table

In older Alder Lake family CPUs with some legacy combinations of BIOS and microcode revisions, it was possible to execute AVX-512 family instructions when disabling all the efficiency cores which do not contain the silicon for AVX-512.

Compilers supporting AVX-512

Assemblers supporting AVX-512

AVX-VNNI, AVX-IFMA

AVX-VNNI is a VEX-coded variant of the AVX512-VNNI instruction set extension. Similarly, AVX-IFMA is a VEX-coded variant of AVX512-IFMA. These extensions provide the same sets of operations as their AVX-512 counterparts, but are limited to 256-bit vectors and do not support any additional features of EVEX encoding, such as broadcasting, opmask registers or accessing more than 16 vector registers. These extensions allow for supporting VNNI and IFMA operations even when AVX-512 is not implemented in the processor.

CPUs with AVX-VNNI

CPUs with AVX-IFMA

AVX-NE-CONVERT

AVX-NE-CONVERT introduces a set of instructions for converting between Bfloat16, half-precision and single-precision floating-point numbers. The new instructions are VEX-coded, and therefore are limited to AVX2 vector registers and lack opmask support from AVX-512. Broadcast is only supported by means of two special instructions for loading numbers from memory, it is not supported as an instruction encoding feature.

CPUs with AVX-NE-CONVERT

AVX-VNNI-INT8, AVX-VNNI-INT16

These instruction sets further extend the AVX-VNNI extension by adding support for more combinations of input data types for the VPDP* series of instructions. Where VPDPBUSD instructions from AVX-VNNI take a vector of unsigned bytes as the first input operand and a vector of signed bytes as the second input operand, AVX-VNNI-INT8 adds variants of these instructions that support signed and unsigned byte inputs at any position. Similarly, where VPDPWSSD from AVX-VNNI take two vectors of signed 16-bit words as input operands, AVX-VNNI-INT16 adds support for signed and unsigned 16-bit word inputs at any position.
For the instructions accepting signed and unsigned inputs, there are distinct instructions for the two possible orders of inputs because VEX encoding only supports the second input operand to be a memory operand. This allows any of the supported data types to be loaded from memory by the instruction.

CPUs with AVX-VNNI-INT8

CPUs with AVX-VNNI-INT16

AVX10

AVX10, announced in July 2023, is a new, "converged" AVX instruction set. It addresses several issues of AVX-512; in particular, that it is split into too many parts. The initial technical paper also made 512-bit vectors optional to support, but as of revision 3.0, vector length enumeration is removed and 512-bit vectors are mandatory.
AVX10 presents a simplified CPUID interface to test for instruction support, consisting of the AVX10 version number. For example, AVX10.2 indicates that a CPU is capable of the second version of AVX10. Initial revisions of the AVX10 technical specifications also included maximum supported vector length as part of the ISA extension name, e.g. AVX10.2/256 would mean a second version of AVX10 with vector length up to 256 bits, but later revisions made that unnecessary.
The first version of AVX10, notated AVX10.1, does not introduce any instructions or encoding features beyond what is already in AVX-512. For CPUs supporting AVX10 and 512-bit vectors, all legacy AVX-512 feature flags will remain set to facilitate applications supporting AVX-512 to continue using AVX-512 instructions.
AVX10.1 was first released in Intel Granite Rapids and AVX10.2 will be available in Diamond Rapids and Nova Lake.

Applications

  • Suitable for floating-point-intensive calculations in multimedia, scientific and financial applications.
  • Increases parallelism and throughput in floating-point SIMD calculations.
  • Reduces register load due to the non-destructive instructions.
  • Improves Linux RAID software performance

Software

  • Cryptography
  • * BSAFE C toolkits uses AVX and AVX2 where appropriate to accelerate various cryptographic algorithms.
  • * OpenSSL uses AVX- and AVX2-optimized cryptographic functions since version 1.0.2. Support for AVX-512 was added in version 3.0.0. Some of these optimizations are also present in various clones and forks, like LibreSSL.
  • Multimedia
  • * Blender uses AVX, AVX2 and AVX-512 in the Cycles render engine.
  • * Native Instruments' Massive X softsynth requires AVX.
  • * AV1 decoder can use AVX2 and AVX-512 on supported CPUs.
  • * AV1 encoder can use AVX2 and AVX-512 to accelerate video encoding.
  • Science, engineering an others
  • * Esri ArcGIS Data Store uses AVX2 for graph storage.
  • * Prime95/MPrime, the software used for GIMPS, started using the AVX instructions since version 27.1, AVX2 since 28.6 and AVX-512 since 29.1.
  • * Einstein@Home uses AVX in some of their distributed applications that search for gravitational waves.
  • * TensorFlow since version 1.6 and tensorflow above versions requires CPU supporting at least AVX.
  • * EmEditor 19.0 and above uses AVX2 to speed up processing.
  • * Microsoft Teams uses AVX2 instructions to create a blurred or custom background behind video chat participants, and for background noise suppression.
  • *, a JSON parsing library, uses AVX2 and AVX-512 to achieve improved decoding speed.
  • *, a library with sorting algorithms for 16, 32 and 64-bit numeric data types, uses AVX2 and AVX-512. The library is used in NumPy and OpenJDK to accelerate sorting algorithms.
  • * Tesseract OCR engine uses AVX, AVX2 and AVX-512 to accelerate character recognition.

Downclocking

Since AVX instructions are wider, they consume more power and generate more heat. Executing heavy AVX instructions at high CPU clock frequencies may affect CPU stability due to excessive voltage droop during load transients. Some Intel processors have provisions to reduce the Turbo Boost frequency limit when such instructions are being executed. This reduction happens even if the CPU hasn't reached its thermal and power consumption limits.
On Skylake and its derivatives, the throttling is divided into three levels:
  • L0 : The normal turbo boost limit.
  • L1 : The "AVX boost" limit. Soft-triggered by 256-bit "heavy" instructions. Hard-triggered by "light" 512-bit instructions.
  • L2 : The "AVX-512 boost" limit. Soft-triggered by 512-bit heavy instructions.
The frequency transition can be soft or hard. Hard transition means the frequency is reduced as soon as such an instruction is spotted; soft transition means that the frequency is reduced only after reaching a threshold number of matching instructions. The limit is per-thread.
In Ice Lake, only two levels persist:
  • L0 : The normal turbo boost limit.
  • L1 : Triggered by any 512-bit instructions, but only when single-core boost is active; not triggered when multiple cores are loaded.
Rocket Lake processors do not trigger frequency reduction upon executing any kind of vector instructions regardless of the vector size. However, downclocking can still happen due to other reasons, such as reaching thermal and power limits.
Downclocking means that using AVX in a mixed workload with an Intel processor can incur a frequency penalty. Avoiding the use of wide and heavy instructions help minimize the impact in these cases. AVX-512VL allows for using 256-bit or 128-bit operands in AVX-512 instructions, making it a sensible default for mixed loads.
On supported and unlocked variants of processors that down-clock, the clock ratio reduction offsets are adjustable and may be turned off entirely via Intel's Overclocking / Tuning utility or in BIOS if supported there.