Quantum processors are routinely qualified by component-level benchmarks — randomized-benchmarking infidelities of one- and two-qubit gates, state-preparation-and-measurement (SPAM) errors, memory errors, crosstalk, and leakage — while their practical capability is measured by system-level benchmarks such as random Clifford circuits with mid-circuit measurements, mirror-benchmarked random circuit sampling (RCS), and layer fidelity. Comparing the two levels currently relies on heuristic gate-counting fits in which an effective depolarizing error is fitted to system data and eyeballed against the component sum. The Quantinuum Helios 98-qubit trapped-ion paper states the gap explicitly: 'a rigorous methodology for comparing component-level to system-level benchmarking performance is an open problem.' The problem is to establish such a methodology — a provably correct, scalable mapping from measured component-level error rates to predicted system-level benchmark outcomes with quantitative bounds and confidence intervals.

Background

A quantum computer's errors are conventionally quantified at two levels. Component-level benchmarks characterize elementary operations in isolation: randomized benchmarking (RB — applying random sequences of Clifford gates and fitting the exponential decay of survival probability, which robustly estimates average gate infidelity) for one- and two-qubit (2Q) gates; SPAM errors from state preparation and measurement; idling or transport-induced memory errors; and error correlations such as crosstalk (unwanted coupling of a gate or measurement on some qubits to others) and leakage (population leaving the computational two-level subspace). System-level benchmarks instead run large, structured-random circuits on the whole processor: scalable protocols such as binary randomized benchmarking (BiRB) and mirror-circuit benchmarking for random Clifford circuits with mid-circuit measurements and resets (MCMRs), layer fidelity / error per layered gate (EPLg) for chains of coupled qubits, and mirror-benchmarked random circuit sampling (RCS — circuits deep enough that sampling their output distribution is believed to be classically intractable, with circuit fidelity inferred from a mirrored, classically simulable copy).

The natural question — do the component numbers predict the system numbers? — matters because users must decide what programs a processor can run, vendors must locate the true performance-limiting errors, and cross-platform comparison needs a defensible yardstick. The answer is empirically 'not reliably': Proctor et al. (Nature Physics 18, 75 (2022)) showed across twelve processors that structured programs fail up to an order of magnitude earlier in size than disordered ones, so standard component error metrics mispredict real-program performance; correlations, non-Markovian effects, and coherent errors (which add amplitudewise rather than probabilitywise, so their net effect is not the product of individual gate fidelities) break naive composition. Current practice bridges the levels with heuristic gate-counting models: e.g., the Helios paper (Ransford et al., arXiv:2511.05465) fits RCS mirror-circuit fidelity to FGC(l)=(1−pspam)N(1−54ϵeff,2Q)N2(l−δ)F_{GC}(l)=(1-p_{\mathrm{spam}})^{N}(1-\frac{5}{4}\epsilon_{\mathrm{eff},2Q})^{\frac{N}{2}(l-\delta)}, where NN is qubit number, ll circuit depth, δ\delta a mirror-circuit boundary correction, and ϵeff,2Q\epsilon_{\mathrm{eff},2Q} a single effective depolarizing 2Q error (a rate at which a gate's output is replaced by the maximally mixed state) aggregating all gate and memory errors. The fitted ϵeff,2Q=2.0(3)×10−3\epsilon_{\mathrm{eff},2Q}=2.0(3)\times10^{-3} agreed with the component-sum prediction 2.2(1)×10−32.2(1)\times10^{-3} within error bars, but the model has no derivation from the measured component channels, no error propagation with coverage guarantees, and no stated class of noise under which it is exact — which is precisely why the authors flag the comparison methodology itself as the open problem.

Rigorous theory exists only in fragments: RB and mirror-benchmarking decay theorems hold under gate-independent or uniform Markovian (memoryless) noise with 2-design twirling — randomizing the applied gates over an ensemble whose second moments match those of uniformly random unitaries, which turns arbitrary noise into an effective stochastic Pauli channel on average — and layer-fidelity protocols inherit direct-RB guarantees per layer, but none of these yields a provable component-to-system map covering crosstalk, leakage, non-Markovianity, and architecture-specific scheduling such as ion shuttling in the quantum charge-coupled device (QCCD) architecture used by Helios, in which ions are physically transported between operation zones.

Problem Statement

Establish a rigorous, scalable methodology for comparing component-level benchmark results of a quantum processor to its system-level benchmark performance. The source (Ransford et al., 'Helios: A 98-qubit trapped-ion quantum computer', arXiv:2511.05465, Sec. III C 1) states the gap verbatim: 'We remark that our method of comparison is heuristic and a rigorous methodology for comparing component-level to system-level benchmarking performance is an open problem.' Concretely, replace the heuristic gate-counting fits currently used, so that system-level quantities are predicted from measured component-level error rates (1Q/2Q randomized-benchmarking infidelities, SPAM errors, transport and idling memory errors, MCMR crosstalk, leakage) with provable guarantees — at minimum for either the fidelity decay of N-qubit random Clifford circuits with mid-circuit measurement and reset or mirror-benchmarked RCS fidelity as a function of depth, the two system-level benchmark families demonstrated in the source, with layer fidelity a further natural target. A complete answer must specify (a) the class of noise processes and assumptions under which the mapping is provably correct, stating explicitly how the error mechanisms aggregated in the source's own accounting — 1Q gates, 2Q gates, memory errors, SPAM errors, MCMR crosstalk, leakage, and correlated and coherent components — are covered or explicitly bounded, with quantitative bounds or confidence intervals relating component estimates to predicted system-level outcomes; and (b) the sampling and computational cost of both the component characterization and the prediction. Comparison against the aggregate system-level results published in the Helios paper (e.g., the fitted ϵeff,2Q\epsilon_{\mathrm{eff},2Q} from random-Clifford and mirrored-RCS data, and the effective MCMR errors) may serve as supporting evidence, but is not a completion requirement; the raw system-level datasets are not publicly released.

References

  1. A. Ransford, M. S. Allman, J. Arkinstall, et al. (Quantinuum and Sandia collaboration), 'Helios: A 98-qubit trapped-ion quantum computer' (2025), arXiv:2511.05465, DOI 10.48550/arXiv.2511.05465, https://arxiv.org/abs/2511.05465
  2. T. Proctor, K. Rudinger, K. Young, E. Nielsen, R. Blume-Kohout, 'Measuring the Capabilities of Quantum Computers', Nature Physics 18, 75-79 (2022), DOI 10.1038/s41567-021-01409-7, arXiv:2008.11294, https://arxiv.org/abs/2008.11294
  3. J. Hines, D. Hothem, R. Blume-Kohout, B. Whaley, T. Proctor, 'Fully scalable randomized benchmarking without motion reversal' (binary randomized benchmarking, BiRB), PRX Quantum 5, 030334 (2024), DOI 10.1103/PRXQuantum.5.030334, arXiv:2309.05147, https://arxiv.org/abs/2309.05147
  4. T. Proctor, S. Seritan, K. Rudinger, E. Nielsen, R. Blume-Kohout, K. Young, 'Scalable randomized benchmarking of quantum computers using mirror circuits', Physical Review Letters 129, 150502 (2022), DOI 10.1103/PhysRevLett.129.150502, arXiv:2112.09853, https://arxiv.org/abs/2112.09853
  5. K. Mayer, A. Hall, T. Gatterman, et al., 'Theory of mirror benchmarking and demonstration on a quantum computer' (2021, revised 2023), arXiv:2108.10431, https://arxiv.org/abs/2108.10431
  6. D. C. McKay, I. Hincks, E. J. Pritchett, M. Carroll, L. C. G. Govia, S. T. Merkel, 'Benchmarking Quantum Processor Performance at Scale' (the layer-fidelity / EPLg protocol) (2023), arXiv:2311.05933, https://arxiv.org/abs/2311.05933
  7. M. J. Lozano Palacio, H. Nayfeh, M. Ware, D. C. McKay, 'Parameter Analysis and Optimization of Layer Fidelity for Quantum Processor Benchmarking at Scale' (2025), arXiv:2510.16915, https://arxiv.org/abs/2510.16915
  8. M. DeCross, R. Haghshenas, M. Liu, et al., 'The computational power of random quantum circuits in arbitrary geometries', Physical Review X 15, 021052 (2025), DOI 10.1103/PhysRevX.15.021052, arXiv:2406.02501, https://arxiv.org/abs/2406.02501
  9. J. M. Lorenz, T. Monz, J. Eisert, et al., 'Systematic benchmarking of quantum computers: status and recommendations' (2025), arXiv:2503.04905, https://arxiv.org/abs/2503.04905
  10. Z. Zimborás, A. Portik, D. Aguirre, et al., 'The EU Quantum Flagship's Key Performance Indicators for Quantum Computing' (2025), arXiv:2512.19653, https://arxiv.org/abs/2512.19653
  11. S. Shirgure, E. Kökcü, A. Mitra, W. A. de Jong, C. Iancu, S. Niu, 'Characterizing and Benchmarking Dynamic Quantum Circuits' (2026), arXiv:2604.03360, https://arxiv.org/abs/2604.03360
  12. D. Hothem, J. Hines, K. Nataraj, R. Blume-Kohout, T. Proctor, 'Predictive Models from Quantum Computer Benchmarks' (2023), arXiv:2305.08796, https://arxiv.org/abs/2305.08796
  13. M. Brieger, F. Krötz, M. Chung, D. Kranzlmüller, 'Evaluating System-Level Fidelity with Peaked Random Circuits' (2026), arXiv:2605.25983, https://arxiv.org/abs/2605.25983
  14. S. K. Seritan, A. Dhumuntarao, A. Q. Wilber-Gauthier, et al., 'Benchmarking quantum computers with any quantum algorithm' (2025), arXiv:2508.05754, https://arxiv.org/abs/2508.05754