Vision processing unit
A vision processing unit is an emerging class of microprocessor; it is a specific type of AI accelerator, designed to accelerate machine vision tasks.
Overview
Vision processing units are distinct from graphics processing units in their suitability for running machine vision algorithms such as CNN, SIFT and similar.They may include direct interfaces to take data from cameras, and have a greater emphasis on on-chip dataflow between many parallel execution units with scratchpad memory, like a spatial architecture or a manycore DSP. But, like video processing units, they may have a focus on low precision fixed point arithmetic for image processing.
Contrast with GPUs
They are distinct from GPUs, which contain specialised hardware for rasterization and texture mapping, and whose memory architecture is optimised for manipulating bitmap images in off-chip memory. VPUs are optimized for performance per watt, while GPUs mainly focus on absolute performance.Target markets are robotics, the internet of things, new classes of digital cameras for virtual reality and augmented reality, smart cameras, and integrating machine vision acceleration into smartphones and other mobile devices.
Examples
- Movidius Myriad X, which is the third-generation vision processing unit in the Myriad VPU line from Intel Corporation.
- Movidius Myriad 2, which finds use in Google Project Tango, Google Clips and DJI drones
- Pixel Visual Core, which is a fully programmable Image, Vision and AI processor for mobile devices
- Microsoft HoloLens, which includes an accelerator referred to as a holographic processing unit, aimed at interpreting camera inputs, to accelerate environment tracking and vision for augmented reality applications.
- Eyeriss, a spatial architecture designed from MIT intended for running convolutional neural networks.
- NeuFlow, a design by Yann LeCun for accelerating convolutions, using a dataflow architecture.
- Mobileye EyeQ, by Mobileye
- Programmable Vision Accelerator, a 7-way VLIW Vision Processor designed by Nvidia.
Broader category
Some processors are not described as VPUs, but are equally applicable to machine vision tasks. These may form a broader category of AI accelerators, however as of 2016 there is no consensus on the name:- IBM TrueNorth, a neuromorphic processor aimed at similar sensor data pattern recognition and intelligence tasks, including video/audio.
- Qualcomm Zeroth Neural processing unit, another entry in the emerging class of sensor/AI oriented chips.
- All models of Intel Meteor Lake processors have a Versatile Processor Unit built-in for accelerating inference for computer vision and deep learning.