RapidIO
RapidIO is a packet-switched interconnect technology used to link electronic components. It allows devices to exchange messages, perform read and write operations, and maintain cache coherence. RapidIO follows common electrical standards, such as those used in Ethernet, and can connect chips, circuit boards, or entire systems together.
History
The RapidIO Trade Association was founded around February 2000 with early members such as Cisco Systems, Galileo Technology, HAL Computer Systems, Lucent Technologies, Mercury Computer Systems, Motorola, Nortel Networks, Seagull Semiconductor, Tundra Semiconductor, and Xilinx. The RapidIO protocol was originally designed by Mercury Computer Systems and Motorola as a replacement for Mercury's RACEway proprietary bus and Freescale's PowerPC bus.Releases
The RapidIO specification revision 1.1, released in March 2001, defined a wide, parallel bus. This version did not achieve broad commercial deployment.The RapidIO specification revision 1.2, released in June 2002, defined a serial interconnection based on the XAUI physical layer. Devices based on this specification saw commercial use in wireless baseband, imaging and military computing.
The RapidIO specification revision 1.3 was released in June 2005.
The RapidIO specification revision 2.0, was released in March 2008. This added more port widths and increased the maximum lane speed to 6.25 GBd / 5 Gbit/s.
The RapidIO specification revision 2.1 was released in September 2009.
The RapidIO specification revision 2.2 was released in May 2011.
The RapidIO specification revision 3.0 released in October 2013. The following changes were made:
- Based on industry-standard Ethernet 10GBASE-KR electrical specifications for short and long reach applications
- Uses the Ethernet 10GBASE-KR DME training scheme for long-reach signal quality optimization
- Defines a 64b/67b encoding scheme to support both copper and optical interconnects and to improve bandwidth efficiency
- Dynamic asymmetric links to save power
- Addition of a time synchronization capability similar to IEEE 1588, but much less expensive to implement
- Support for 32-bit device IDs, increasing maximum system size and enabling hardware virtualization support
- Revised routing table programming model simplifies network management software
- Packet exchange protocol optimizations
- MECS Time Synchronization protocol for smaller embedded systems. MECS Time Synchronization supports redundant time sources. This protocol is lower cost than the Timestamp Synchronization Protocol introduced in revision 3.0
- PRBS test facilities and standard register interface.
- Structurally Asymmetric Link behavioral definition and standard register interface. These structurally Asymmetric Links carry much more data in one direction than the other, for applications such as sensors or processing pipelines. Unlike dynamic asymmetric links, Structurally Asymmetric Links allow implementers to remove lanes on boards and in silicon, saving size, weight, and power. Structurally asymmetric links also allow the use of alternative lanes in the case of a hardware failure on a multi-lane port.
- Extended error log to capture a series of errors for diagnostic purposes
- Space device profiles for endpoints and switches, which define what it means to be a space-compliant RapidIO device.
The RapidIO specification revision 4.0 was released in June 2016. It had the following changes compared to the 3.x specifications:
- Support 25 Gbaud lane rate and physical layer specification, with associated programming model changes
- Allow IDLE3 to be used with any Baud Rate Class, with specified IDLE sequence negotiation
- Increased maximum packet size to 284 bytes in anticipation of Cache Coherency specification
- Support 16 physical layer priorities
- Support “Error Free Transmission” for high throughput isochronous information transfer
Wireless infrastructure
RapidIO fabrics are used in cellular infrastructure, including 3G, 4G, and LTE networks. RapidIO fabrics were originally designed to support connecting different types of microprocessors from different manufacturers together in a single system.Data centers
Systems for high‑performance computing and data analytics have been deployed using the RapidIO fabric as a unified intra‑system interconnect.Also, using an open modular data center and compute platform, a heterogeneous HPC system has showcased the low latency attribute of RapidIO to enable real-time analytics. In March 2015 a top-of-rack switch was announced to drive RapidIO into mainstream data center applications.
Aerospace
The interconnect or "bus" is a key technology in the design of spacecraft avionics systems, influencing system architecture and complexity. Existing commercial protocols, including RapidIO, InfiniBand, Fibre Channel, and Ethernet, have been considered for use in spacecraft interconnects.The Next Generation Spacecraft Interconnect Standard is a modular standards initiative for spacecraft interconnects that supports integration of multiple processor types and high-performance communication within spacecraft avionics systems.
PHY roadmap
The RapidIO roadmap aligns with Ethernet PHY development. RapidIO specifications for 50 GBd and higher links are under investigation.Terminology
; Link Partner: One end of a RapidIO link.; Endpoint: A device that can originate and/or terminate RapidIO packets.
; Processing Element: A device which has at least one RapidIO port
; Switch: A device that can route RapidIO packets.
Protocol overview
The RapidIO protocol is defined in a 3-layered specification:- Physical: Electrical specifications, PCS/PMA, link-level protocol for reliable packet exchange
- Transport: Routing, multicast, and programming model
- Logical: Logical I/O, messaging, global shared memory, flow control, data streaming
- System Initialization
- Error Management/Hot Swap
Physical layer
- XAUI for lane speeds of 1.25, 2.5, and 3.125 GBd
- OIF CEI 6+ Gbit/s for lane speeds of 5.0 and 6.25 GBd
- 10GBASE-KR 802.3-ap and 802.3-ba for lane speeds of 10.3125 GBd
- 8b/10b for lane speeds up to 6.25 GBd
- 64b/67b, similar to that used by Interlaken for lane speeds over 6.25 GBd
Packets
Every packet has two values that control the physical layer exchange of that packet. The first is an acknowledge ID, which is the link-specific, unique, 5-, 6-, or 12-bit value that is used to track packets exchanged on a link. Packets are transmitted with serially increasing ackID values. Because the ackID is specific to a link, the ackID is not covered by CRC, but by protocol. This allows the ackID to change with each link it passes over, while the packet CRC can remain a constant end-to-end integrity check of the packet. When a packet is successfully received, it is acknowledged using the ackID of the packet. A transmitter must retain a packet until it has been successfully acknowledged by the link partner.The second value is the packet's physical priority. The physical priority is composed of the Virtual Channel identifier bit, the Priority bits, and the Critical Request Flow bit. The VC bit determines if the Priority and CRF bits identify a Virtual Channel from 1 to 8, or are used as the priority within Virtual Channel 0. Virtual Channels are assigned guaranteed minimum bandwidths. Within Virtual Channel 0, packets of higher priority can pass packets of lower priority. Response packets must have a physical priority higher than requests in order to avoid deadlock.
The physical layer contribution to RapidIO packets is a 2-byte header at the beginning of each packet that includes the ackID and physical priority, and a final 2-byte CRC value to check the integrity of the packet. Packets larger than 80 bytes also have an intermediate CRC after the first 80 bytes. With one exception a packet's CRC value acts as an end-to-end integrity check.
Control symbols
RapidIO control symbols can be sent at any time, including within a packet. This gives RapidIO the lowest possible in-band control path latency, enabling the protocol to achieve high throughput with smaller buffers than other protocols.Control symbols are used to delimit packets, to acknowledge packets, reset and to distribute events within the RapidIO system. Control symbols are also used for flow control and for error recovery.
The error recovery procedure is very fast. When a receiver detects a transmission error in the received data stream, the receiver causes its associated transmitter to send a Packet Not Accepted control symbol. When the link partner receives a Packet Not Accepted control symbol, it stops transmitting new packets and sends a Link Request/Port Status control symbol. The Link Response control symbol indicates the ackID that should be used for the next packet transmitted. Packet transmission then resumes.