Data processing unit
A data processing unit is a programmable computer processor that tightly integrates a general-purpose CPU with network interface hardware. Sometimes they are called "IPUs" or "SmartNICs". They can be used in place of traditional NICs to relieve the main CPU of complex networking responsibilities and other "infrastructural" duties; although their features vary, they may be used to perform encryption/decryption, serve as a firewall, handle TCP/IP, process HTTP requests, or even function as a hypervisor or storage controller. These devices can be attractive to cloud computing providers whose servers might otherwise spend a significant amount of CPU time on these tasks, cutting into the cycles they can provide to guests.
Large-scale AI data centers are an emerging use case for DPUs. In these environments, massive amounts of data must be moved rapidly among CPUs, GPUs, and storage systems to handle complex AI workloads. By offloading tasks such as packet processing, encryption, and traffic management, DPUs help reduce latency and improve energy efficiency, enabling these data centers to maintain the high throughput and scalability needed for advanced machine learning operations.
Alongside their role in accelerating network and storage functions, some vendors and cloud providers describe DPUs as a "third pillar of computing," reflecting their growing role in modern data-center architectures. Unlike traditional processors, a DPU typically resides on a network interface card, allowing data to be processed at the network’s line rate before it reaches the CPU. This approach offloads critical but lower-level system duties—such as security, load balancing, and data routing—from the central processor, thus freeing CPUs and GPUs to focus on application logic and AI-specific computations.