NVLink


NVLink is a wire-based serial, multi-lane, near-range, communications link developed by Nvidia. Unlike PCI Express, a device can consist of multiple NVLinks, and devices can use mesh networking to communicate instead of a central hub/switch. The protocol was first announced in March 2014 and uses a proprietary high-speed signaling interconnect.
For small numbers of GPUs, the NVLink lanes on a single device are sufficient for an all-to-all mesh connectivity. To accommodate higher GPU counts, NVLink since 2018 use a packet-switched architecture, where a central switch can serve up to 32 two-lane ports. The NVSwitch for NVLink 4.0 can produce some simple computation of its own to reduce the need for communication thanks to the "SHARP" accelerator.

Principle

NVLink is developed by Nvidia for data and control code transfers in processor systems between CPUs and GPUs and between GPUs and GPUs. NVLink specifies a point-to-point connection with data rates of 20, 25, and 50 Gbit/s per differential pair. For NVLink 1.0 and 2.0, eight differential pairs form a "sub-link", and two "sub-links", one for each direction, form a "link". Starting from NVlink 3.0, only four differential pairs form a "sub-link". For NVLink 2.0 and higher, the total data rate for a sub-link is 25 GB/s, and the total data rate for a link is 50 GB/s. Each V100 GPU supports up to six links. Thus, each GPU is capable of supporting up to 300 GB/s in total bi-directional bandwidth. NVLink products introduced to date focus on the high-performance application space. Announced May 14, 2020, NVLink 3.0 increases the data rate per differential pair from 25 Gbit/s to 50 Gbit/s while decreasing the number of pairs per NVLink from 8 to 4. With 12 links for an Ampere-based A100 GPU, this brings the total bandwidth to 600 GB/s. The Hopper GPU microarchitecture, announced in March 2022, has 18 NVLink 4.0 links, enabling a total bandwidth of 900 GB/s. Thus, NVLink 2.0, 3.0, and 4.0 all have a 50 GB/s per bidirectional link data rate, and have 6, 12, and 18 links, correspondingly.

Performance

The following table shows a basic metrics comparison based on standard specifications:
Inter­connectTransfer rateLine codeModulationEffective payload rate per lane or NVLink Max. total lane length Total Links Total Bandwidth Realized in design
PCIe 3.x8 GT/s128b/130bNRZ0.99 GB/s31.51 GB/sPascal, Volta, Turing
PCIe 4.016 GT/s128b/130bNRZ1.97 GB/s63.02 GB/sVolta on Xavier, Ampere, POWER9
PCIe 5.032 GT/s128b/130bNRZ3.94 GB/s126.03 GB/sHopper
PCIe 6.064 GT/s236B/256B FLITPAM4 FEC7.56 GB/s242 GB/sBlackwell
NVLink 1.020 GT/sNRZ20 GB/s4160 GB/sPascal, POWER8+
NVLink 2.025 GT/sNRZ25 GB/s6300 GB/sVolta, POWER9
NVLink 3.050 GT/sNRZ25 GB/s12600 GB/sAmpere
NVLink 4.050 GT/sPAM4 differential-pair25 GB/s18900 GB/sHopper, Nvidia Grace
NVLink 5.0100 GT/sPAM4 differential-pair50 GB/s181800 GB/sBlackwell, Nvidia Grace

The following table shows a comparison of relevant bus parameters for real world semiconductors offering NVLink as one of their options:
SemiconductorBoard/bus delivery variantInterconnectTransmission technology rate Lanes per sub-link Sub-link data rate Sub-link or unit countTotal data rate Total lanes Total data rate
Nvidia GP100P100 SXM, P100 PCI-EPCIe 3.08 GT/s16 + 16 128 Gbit/s = 16 GB/s116 + 16 GB/s32 32 GB/s
Nvidia GV100V100 SXM2, V100 PCI-EPCIe 3.08 GT/s16 + 16 128 Gbit/s = 16 GB/s116 + 16 GB/s32 32 GB/s
Nvidia TU104GeForce RTX 2080, Quadro RTX 5000PCIe 3.08 GT/s16 + 16 128 Gbit/s = 16 GB/s116 + 16 GB/s32 32 GB/s
Nvidia TU102GeForce RTX 2080 Ti, Quadro RTX 6000/8000PCIe 3.08 GT/s16 + 16 128 Gbit/s = 16 GB/s116 + 16 GB/s32 32 GB/s
Nvidia GA100
Nvidia GA102		Ampere A100 PCIe 4.016 GT/s16 + 16 256 Gbit/s = 32 GB/s132 + 32 GB/s32 64 GB/s
Nvidia GP100P100 SXM, NVLink 1.020 GT/s8 + 8 160 Gbit/s = 20 GB/s480 + 80 GB/s64160 GB/s
Nvidia GV100V100 SXM2 NVLink 2.025 GT/s8 + 8 200 Gbit/s = 25 GB/s6150 + 150 GB/s96300 GB/s
Nvidia TU104GeForce RTX 2080, Quadro RTX 5000NVLink 2.025 GT/s8 + 8 200 Gbit/s = 25 GB/s125 + 25 GB/s1650 GB/s
Nvidia TU102GeForce RTX 2080 Ti, Quadro RTX 6000/8000NVLink 2.025 GT/s8 + 8 200 Gbit/s = 25 GB/s250 + 50 GB/s32100 GB/s
Nvidia GA100Ampere A100 NVLink 3.050 GT/s4 + 4 200 Gbit/s = 25 GB/s12300 + 300 GB/s96600 GB/s
Nvidia GA102GeForce RTX 3090, Quadro RTX A6000NVLink 3.028.125 GT/s4 + 4 112.5 Gbit/s = 14.0625 GB/s456.25 + 56.25 GB/s16112.5 GB/s
NVSwitch for HopperNVLink 4.0106.25 GT/s9 + 9 450 Gbit/s183600 + 3600 GB/s1287200 GB/s
Nvidia Grace CPUNvidia GH200 SuperchipPCIe-5 @ 512 GB/s-------
Nvidia Grace CPUNvidia GH200 SuperchipNVLink-C2C @ 900 GB/s-------
Nvidia Hopper GPUNvidia GH200 SuperchipNVLink-C2C @ 900 GB/s-------
Nvidia Hopper GPUNvidia GH200 SuperchipNVLink 4 @ 900 GB/s-------

Real world performance could be determined by applying different data transmission overhead costs, as well as usage rates. Those come from various sources:
Those physical limitations usually reduce the data rate to between 90-95 percent of the transfer rate. NVLink benchmarks show an achievable transfer rate of about 35.3 Gbit/s for a 40 Gbit/s NVLink connection towards a P100 GPU in a system that is driven by a set of IBM POWER8 CPUs.

Usage with plug-in boards

For the various versions of plug-in boards that expose extra connectors for joining them into a NVLink group, a similar number of slightly varying, relatively compact, PCB based interconnection plugs does exist. Typically only boards of the same type will mate together due to their physical and logical design. For some setups two identical plugs need to be applied for achieving the full data rate. As of now the typical plug is U-shaped with a fine grid edge connector on each of the end strokes of the shape facing away from the viewer. The width of the plug determines how far away the plug-in cards need to be seated to the main board of the hosting computer system - a distance for the placement of the card is commonly determined by the matching plug. The interconnect is often referred as Scalable Link Interface from 2004 for its structural design and appearance, even if the modern NVLink based design is of a quite different technical nature with different features in its basic levels compared to the former design. Reported real world devices are:
  • Quadro GP100
  • Quadro GV100
  • GeForce RTX 2080 based on TU104
  • GeForce RTX 2080 Ti based on TU102
  • GeForce RTX 3090 based on GA102
  • Quadro RTX 5000 based on TU104
  • Quadro RTX 6000 based on TU102
  • Quadro RTX 8000 based on TU102

    Service software and programming

For the Tesla, Quadro and Grid product lines, the NVML-API offers a set of functions for programmatically controlling some aspects of NVLink interconnects on Windows and Linux systems, such as component evaluation and versions along with status/error querying and performance monitoring. Further, with the provision of the NCCL library, developers in the public space shall be enabled for realizing, e.g., powerful implementations for artificial intelligence and similar computation hungry topics atop NVLink. The page "3D Settings" » "Configure SLI, Surround, PhysX" in the Nvidia Control panel and the CUDA sample application "simpleP2P" use such APIs to realize their services in respect to their NVLink features. On the Linux platform, the command line application with sub-command "nvidia-smi nvlink" provides a similar set of advanced information and control.