Circuit Layer Operations per Second


Circuit Layer Operations per Second is a benchmarking metric of quantum computers developed by IBM in 2021. While Quantum volume is more indicative of a qubit's performance and quality, CLOPS is a measure of a quantum computer’s execution speed. CLOPS incorporates both the time to run a circuit as well as the real-time and near-time classical computation, enabling it to serve as a holistic single measure of speed. CLOPS in conjunction with the number of qubits and the quantum volume was originally envisioned by IBM to be the 3 core metrics for comparing quantum devices.''''''

Redefinition

Originally CLOPS was designed to be related to quantum volume, in the sense that each circuit layer was idealized square quantum volume circuits. However, in 2023 IBM created a new definition of CLOPS which is related to a specific hardware layer. The different versions are differentiated with a subscript: CLOPSv for virtual and CLOPSh for hardware. Functionally, the core difference is that instead of using quantum volume in the calculation, a different metric known as layer fidelity is used instead, measured by the error per layered gate protocol. This enables different hardware with different quantum volumes to be more accurately compared.

Mathematical explanation of the benchmark

  • Define template circuits; each one composed of different quantum circuits
  • is the number of shots to run
  • is the quantum volume as log2
  • is the total amount of time in seconds taken for execution, including the data transfer, transpilation, measurements, resets, and delays
CLOPSv =
The main update in CLOPSh was to replace the quantum volume metric D above with the something IBM termed the layer fidelity, which according to IBM "provides a benchmark that encapsulates the entire processor’s ability to run circuits while revealing information about individual qubits, gates, and crosstalk." Layer fidelity is measured by the error per layered gate. This inflates the CLOPS score for quantum computers that are sparsely connected.

Table of measured CLOPS