Geometric origin of the quaternion determinant in circular-ensemble eigenvalue correlations (Dyson's desideratum)

Dyson's 1970 paper established that the n-point correlation functions of the three circular ensembles — the symmetry classes with Dyson index β = 1 (orthogonal), β = 2 (unitary) and β = 4 (symplectic), realized as the compact symmetric spaces U(N)/O(N), U(N) and U(2N)/Sp(2N) with their invariant measures — are given by quaternion determinants of self-dual 2×2 matrix kernels for β = 1, 4 and by ordinary determinants for β = 2, but proved this by algebraic verification. Dyson remarked that the proof was 'a mere verification' and that it would be highly desirable to find a more illuminating proof in which the appearance of the quaternion determinant is related directly to the structure of the symmetric space. The problem is to produce such a structural derivation: obtain the quaternion-determinant (equivalently, Pfaffian) form of the correlation functions, including the explicit kernels with their step-function components, directly from the geometry and invariant integration on the symmetric spaces. Known partial routes — Pfaffian and skew-orthogonal-polynomial reformulations, supersymmetric methods over symmetric superspaces, symmetric-space derivations of joint eigenvalue densities, and Cartan-classification programs — reorganize or extend the result, but the quaternion-determinant correlation-kernel structure has not yet been obtained from the symmetric-space geometry, and producing that derivation is the problem.

Background

Random matrix theory models complex spectra by ensembles of large matrices with specified symmetries. In Dyson's standard symmetry classification, a Hamiltonian with time-reversal symmetry and no spin-orbit coupling is modeled by real symmetric matrices (orthogonal class, Dyson index β = 1); breaking time-reversal symmetry gives complex Hermitian matrices (unitary class, β = 2); and time-reversal symmetry with $T^2 = -1$ gives matrices over the real quaternions that are self-dual, meaning equal to their quaternionic conjugate transpose (symplectic class, β = 4). The circular ensembles place the eigenvalues on the unit circle: the circular orthogonal ensemble (COE) is the set of symmetric unitary matrices, the circular unitary ensemble (CUE) is the unitary group $U(N)$ with Haar measure, and the circular symplectic ensemble (CSE) is the set of self-dual unitary quaternion matrices. Each of these spaces is a compact Riemannian symmetric space — a manifold of the form $G/K$, where $K$ is the fixed-point subgroup of an involution of a Lie group $G$ — namely $U(N)/O(N)$, $U(N)$ itself, and $U(2N)/\mathrm{Sp}(2N)$ respectively, and each carries its invariant measure. The joint density of the $N$ eigenangles $\theta_j$ is $C_{N\beta}|\Delta|^\beta$, where $\Delta = \prod_{j<k}(e^{i\theta_j} - e^{i\theta_k})$ and $C_{N\beta}$ is a normalization constant.

In his 1970 paper Dyson computed the n-angle correlation functions $R_{Nn\beta}$ (the marginal densities obtained by integrating out $N-n$ eigenangles) for all $n$ and $N$. For β = 2 he recovered the determinantal form with the scalar kernel $s_N(\theta) = \sin(N\theta/2)/(2\pi\sin(\theta/2))$; for β = 1 and β = 4 the correlations are quaternion determinants, $R_{Nn\beta} = \operatorname{QDet}[\sigma_{N\beta}(\theta_i-\theta_j)]$, of self-dual matrices whose entries are the 2×2 quaternion (equivalently complex-matrix) kernels

$$\sigma_{N1}(\theta) = \begin{bmatrix} s_N(\theta) & Ds_N(\theta) \\ Js_N(\theta) & s_N(\theta) \end{bmatrix}, \qquad \sigma_{N4}(\theta) = \tfrac{1}{2}\begin{bmatrix} s_{2N}(\theta) & Ds_{2N}(\theta) \\ Is_{2N}(\theta) & s_{2N}(\theta) \end{bmatrix},$$

where $Ds_N = ds_N/d\theta$, $Is_N(\theta) = \int_0^\theta s_N(\theta')\,d\theta'$, and $Js_N = Is_N - \varepsilon_N$, where $\varepsilon_N$ is the parity-dependent step function $\varepsilon_N(\theta) = \tfrac{1}{2}(-1)^m$ for $2\pi m < \theta < 2\pi(m+1)$ when $N$ is even and $\varepsilon_N(\theta) = m + \tfrac{1}{2}$ when $N$ is odd, with $m$ an integer labeling the interval and $\varepsilon_N(2\pi m) = 0$ for $N$ even and $\varepsilon_N(2\pi m) = m$ for $N$ odd. Here the quaternion determinant (Dyson determinant) of a self-dual $n\times n$ quaternion matrix $M$ is $\operatorname{QDet} M = \sum_P (-1)^{n-\ell}\prod (M_{ab}M_{bc}\cdots M_{sa})$, the sum over permutations $P$ decomposed into $\ell$ cycles; it is equivalent to the Pfaffian (the square-root of the determinant of an antisymmetric matrix) via $(\operatorname{QDet} M)^2 = \det A(M)$, where $A(M)$ is the $2n\times 2n$ complex matrix obtained by expanding each quaternion entry as a 2×2 block — an identity Dyson proved. In the bulk scaling limit $N \to \infty$ the kernels converge to expressions built on the sine kernel $s(r) = \sin(\pi r)/(\pi r)$, giving the universal local statistics of the three classes.

Dyson's proof that these formulas hold proceeds by verification: he first proves the normalization identity (the quaternion determinant over the full set of $N$ angles equals the joint density $C_{N\beta}|\Delta|^\beta$, by determinant manipulations, with an infinitesimal δ-regularization needed in the odd-$N$ orthogonal case), and then an integration recurrence — integrating the quaternion determinant over one angle returns the quaternion determinant of size one less (now called the Dyson integration theorem) — which together force the quaternion-determinant functions to coincide with the correlation functions. Dyson himself flagged the conceptual gap: as quoted by Zirnbauer (1996), the proof 'is a mere verification', and it 'would be highly desirable to find a more illuminating proof, in which the appearance of the [final result] might be related directly to the structure of the symmetric space', the bracket elision being Zirnbauer's editorial one and standing for the quaternion-determinant representation. The question thus sits at the junction of random matrix theory, harmonic analysis on symmetric spaces (invariant integration; KAK-type decompositions, meaning the Cartan decompositions that write each element of $G$ as $k_1 a k_2$ with $k_1, k_2 \in K$; and calculus with radial parts and zonal spherical functions, the $K$-bi-invariant eigenfunctions of the invariant differential operators on $G/K$), and the theory of determinantal and Pfaffian point processes (random point configurations whose correlation functions are determinants, respectively Pfaffians, of a kernel).

Problem Statement

Produce the 'more illuminating proof' Dyson asked for in his 1970 paper: a derivation of the n-point correlation functions of the circular orthogonal and circular symplectic ensembles — with the circular unitary ensemble as the scalar reference case — in which their quaternion-determinant form is obtained directly from the structure of the underlying symmetric spaces and their invariant measures, rather than by the algebraic verification route (normalization identity plus the Dyson integration-theorem recurrence) used in the original paper. Concretely, the derivation should start from the symmetric-space realizations $U(N)/O(N)$ for the circular orthogonal ensemble (β = 1) and $U(2N)/\mathrm{Sp}(2N)$ for the circular symplectic ensemble (β = 4), together with their invariant measures, and should end with the quaternion-determinant (equivalently Pfaffian) formula for the n-angle correlation functions, with the explicit self-dual kernels whose entries are built from the finite-N sine-type kernel $s_N(\theta) = \sin(N\theta/2)/(2\pi\sin(\theta/2))$, its derivative $Ds_N$, its integral $Is_N$, and $Js_N = Is_N - \varepsilon_N$ with $\varepsilon_N$ the parity-dependent step function ($\varepsilon_N(\theta) = \tfrac{1}{2}(-1)^m$ for $2\pi m < \theta < 2\pi(m+1)$ when $N$ is even, $\varepsilon_N(\theta) = m + \tfrac{1}{2}$ when $N$ is odd, $m$ an integer, with the endpoint values as stated in the Background), with $s_{2N}$ replacing $s_N$ for β = 4 at finite $N$, and with the bulk scaling limits built on $s(r) = \sin(\pi r)/(\pi r)$ (argument doubled for β = 4). Each kernel component — including the distributional step-function entry — should be identified with a geometric or representation-theoretic object of the symmetric space (for example, objects of the invariant-integration, KAK-decomposition, or zonal-spherical-function calculus on $G/K$), and the emergence of the quaternionic structure should be a consequence of the geometry, explaining in particular why the orthogonal and symplectic classes carry 2×2 matrix kernels while the unitary class carries a scalar kernel. The derivation must be genuinely structural: its load-bearing steps must be invariant integration and harmonic analysis on $G/K$ rather than a re-labeled verification, and the kernels must be obtained by invariant integration on $G/K$, with each kernel component identified with a geometric or representation-theoretic object of that space; neither the normalization identity nor the Dyson integration recurrence may serve as its main engine. A precise proof that no such geometric derivation exists, or that the question is ill-posed, also resolves the task.

References

1. Freeman J. Dyson, "Correlations between eigenvalues of a random matrix", Communications in Mathematical Physics 19(3), 235-250 (1970). DOI 10.1007/BF01646824, https://doi.org/10.1007/BF01646824
2. M. L. Mehta, "A note on correlations between eigenvalues of a random matrix", Communications in Mathematical Physics 20, 245-250 (1971). DOI 10.1007/BF01646557, https://doi.org/10.1007/BF01646557
3. Martin R. Zirnbauer, "Riemannian symmetric superspaces and their origin in random-matrix theory", Journal of Mathematical Physics 37, 4986-5018 (1996). DOI 10.1063/1.531675, arXiv math-ph/9808012, https://arxiv.org/abs/math-ph/9808012
4. Martin R. Zirnbauer, "Supersymmetry for systems with unitary disorder: circular ensembles", Journal of Physics A: Mathematical and General 29, 7113-7136 (1996). DOI 10.1088/0305-4470/29/22/013, https://doi.org/10.1088/0305-4470/29/22/013
5. Eduardo Dueñez, "Random Matrix Ensembles Associated to Compact Symmetric Spaces", Communications in Mathematical Physics 244, 29-61 (2003). DOI 10.1007/s00220-003-0994-2, arXiv math-ph/0111005, https://arxiv.org/abs/math-ph/0111005
6. Jinpeng An, Zhengdong Wang, Kuihua Yan, "A Generalization of Random Matrix Ensemble I: General Theory", Pacific Journal of Mathematics 228(1), 1-17 (2006). DOI 10.2140/pjm.2006.228.1, arXiv math-ph/0502020, https://arxiv.org/abs/math-ph/0502020
7. Jinpeng An, Zhengdong Wang, Kuihua Yan, "A Generalization of Random Matrix Ensemble II: Concrete Examples and Integration Formulae", arXiv math-ph/0502021 (2005), https://arxiv.org/abs/math-ph/0502021
8. M. Caselle, U. Magnea, "Random matrix theory and symmetric spaces", Physics Reports 394, 41-156 (2004). DOI 10.1016/j.physrep.2003.12.004, arXiv cond-mat/0304363, https://arxiv.org/abs/cond-mat/0304363
9. Taro Nagao, "Pfaffian Expressions for Random Matrix Correlation Functions", Journal of Statistical Physics 129, 1137-1158 (2007). DOI 10.1007/s10955-007-9415-9, arXiv 0708.2036, https://arxiv.org/abs/0708.2036
10. Gernot Akemann, Eugene Kanzieper, "Integrable Structure of Ginibre's Ensemble of Real Random Matrices and a Pfaffian Integration Theorem", Journal of Statistical Physics 129, 1159-1231 (2007). DOI 10.1007/s10955-007-9381-2, arXiv math-ph/0703019, https://arxiv.org/abs/math-ph/0703019
11. Alexei Borodin, Eugene Kanzieper, "A Note on the Pfaffian Integration Theorem", Journal of Physics A: Mathematical and Theoretical 40, F849-F855 (2007). DOI 10.1088/1751-8113/40/36/F01, arXiv 0707.2784, https://arxiv.org/abs/0707.2784
12. Peter J. Forrester, Anthony Mays, "A method to calculate correlation functions for β=1 random matrices of odd size", Journal of Statistical Physics 134, 443-462 (2009). DOI 10.1007/s10955-009-9684-6, arXiv 0809.5116, https://arxiv.org/abs/0809.5116
13. Vladislav Kargin, "On Pfaffian random point fields", Journal of Statistical Physics 154, 681-704 (2014). DOI 10.1007/s10955-013-0900-z, arXiv 1210.6603, https://arxiv.org/abs/1210.6603
14. Martin R. Zirnbauer, "Symmetry Classes", in The Oxford Handbook of Random Matrix Theory (Oxford University Press, 2015), pp. 42-65. DOI 10.1093/oxfordhb/9780198744191.013.3, arXiv 1001.0722, https://arxiv.org/abs/1001.0722
15. Anthony Mays, "A geometrical triumvirate of real random matrices" (PhD thesis, University of Melbourne, submitted November 2011), arXiv 1202.1218, https://arxiv.org/abs/1202.1218
16. Persi Diaconis, Peter J. Forrester, "A. Hurwitz and the origins of random matrix theory in mathematics", Random Matrices: Theory and Applications 6(1), 1730001 (2017). DOI 10.1142/S2010326317300017, arXiv 1512.09229, https://arxiv.org/abs/1512.09229
17. Alan Edelman, Sungwoo Jeong, "On the Cartan decomposition for classical random matrix ensembles", Journal of Mathematical Physics 63, 061705 (2022). DOI 10.1063/5.0087010, arXiv 2011.08087, https://arxiv.org/abs/2011.08087