In 1945, Dirac defined a 'formal probability' that non-commuting observables take given values — the object now called the Kirkwood-Dirac distribution — which reproduces the classical expectation-value calculus but is complex in general and so lacks a direct physical interpretation as a probability. He judged the classical-quantum analogy it anchors incomplete: the generating function of the quantum contact transformation is complex, constrained systems introduce extra variables λn\lambda_n that are not ordinary observables, the construction presumes complete eigenbases, and non-commuting relativistic observables at a single space-time point cannot both enter a function. The problem is to complete the classical-quantum analogy by giving the complex formal probability a physically satisfactory interpretation. Later work settled operational access — the distribution is directly measurable via weak values, propagates by Bayes' rule, and is uniquely characterized axiomatically — and proved that no genuine joint probability with correct marginals and state-linearity exists, so non-real values are ineliminable. What these complex values physically mean, and how the analogy extends to trajectories, constrained systems, and space-time, remains actively contested without consensus.

Background

In classical mechanics, the state of a particle is a point (q,p)(q,p) in phase space, every observable is a function of that point, and statistical statements are ordinary probabilities obeying the standard axioms of probability theory. In quantum mechanics, observables are Hermitian operators (operators with real eigenvalues) acting on a Hilbert space, and the operators representing conjugate quantities such as position qq and momentum pp do not commute: qp≠pqqp \neq pq, which underlies Heisenberg's uncertainty principle. Non-commuting observables have no common set of eigenvectors, so no genuine joint probability distribution over their simultaneous values can reproduce all quantum statistics. This incompatibility is one of the sharpest differences between the classical and quantum pictures and the main obstacle to any literal phase-space formulation of quantum theory.

Dirac's 1945 paper 'On the Analogy Between Classical and Quantum Mechanics' argued that non-commutation is essentially the only difference between the two theories, and set out to develop the analogy systematically. He first defined general functions of non-commuting observables — 'well-ordered' functions constructed by substituting eigenvalues one observable at a time, in a fixed order, using each observable's eigenvectors. Choosing the function to be the indicator of a set of values then defines a joint quantity for non-commuting observables: a 'formal probability' PP given by the expectation, in the state ∣X⟩|X\rangle, of the ordered product of the projectors onto the chosen eigenvalues (a projector is the Hermitian operator that projects onto the eigenspace of a given value). This PP obeys the classical rule that expectation values are averages over the distribution, ⟨f⟩=∑fP\langle f\rangle = \sum f P, exactly as in classical statistics — but PP is in general a complex number, so it cannot be read as a probability. Applied to position at successive times, PP becomes a probability-like weight for trajectories, built from transformation functions, the matrix elements ⟨q′∣Q′⟩\langle q'|Q'\rangle that connect two representations. Dirac further showed that if SS is defined by ⟨q′∣Q′⟩=eiS/ℏ\langle q'|Q'\rangle = e^{iS/\hbar}, then the conjugate momenta satisfy p=∂S/∂qp = \partial S/\partial q and PQ=−∂S/∂QP_Q = -\partial S/\partial Q — precisely the equations of a classical contact transformation, a canonical change of coordinates generated by a function SS — so quantum transformation theory mirrors classical mechanics, but with a generally complex generating function. When ⟨q′∣Q′⟩\langle q'|Q'\rangle vanishes on a constraint surface Fn(q,Q)=0F_n(q,Q)=0, extra terms appear involving new variables λn\lambda_n that behave like classical Lagrange multipliers but are singular objects built from derivatives of delta functions. In the relativistic discussion, the chosen ordering of two observables is Lorentz invariant only when they lie outside each other's light cones; two non-commuting observables referring to exactly the same space-time point cannot both enter a function.

Dirac's own closing assessment records these difficulties. The formal probability is complex, so it does not have an immediate physical interpretation as a probability; only when it is close to zero can it be taken to indicate that the corresponding values are unlikely. The generating function SS is not generally real, the λn\lambda_n are not expressible as ordinary observables in the original coordinates, the construction presumes the observables have complete sets of eigenvectors, and the relativistic same-point restriction stands. Hence, in his judgment, further development was needed to realize the classical-quantum analogy in a physically satisfactory manner.

The object Dirac defined in 1945 is now a standard structure. It had been introduced earlier by Kirkwood (1933) in a statistical-mechanics context and repeatedly rediscovered since; it is called the Kirkwood-Dirac (KD) distribution. For a density operator ρ\rho (the general description of a mixed quantum state) and two observables with projectors Πa\Pi_a and Πb\Pi_b, the KD distribution is QKD(a,b)=Tr[ΠbρΠa]Q_{\mathrm{KD}}(a,b) = \mathrm{Tr}[\Pi_b \rho \Pi_a] together with its conjugate ordering Tr[ΠaρΠb]\mathrm{Tr}[\Pi_a \rho \Pi_b]; its real part is the Terletsky-Margenau-Hill distribution. It is a quasiprobability: a distribution that reproduces the correct Born-rule probabilities for each observable separately (its marginals; the Born rule assigns measurement probabilities from the state's weights on the eigenvectors) but can take negative or non-real values, like the more familiar Wigner function for position and momentum.

Later work transformed the status of this object. Weak values — the complex conditioned averages that Aharonov, Albert and Vaidman (1988) showed arise when a meter is coupled weakly between a state preparation and a later post-selected measurement outcome — gave the same complex quantities an operational meaning: the real part of a weak value shifts a pointer's position and the imaginary part its momentum. Bamber and Lundeen (2014) directly measured Dirac's distribution for a photon's transverse position and momentum, showing that it completely determines the quantum state and propagates according to Bayes' rule despite not being a probability. Hofmann (2011-2014) recast complex joint and conditional probabilities as the deterministic relations between the outcomes of different measurements, and gave axiomatic conditions singling out the ordered-product definition among all candidate joint 'probabilities'. Conversely, impossibility results now pin down what cannot be done: Lostagno et al. (2023) proved that for non-commuting observables no genuine joint probability simultaneously has correct marginals and is linear in the state, so keeping both properties forces quasiprobabilities with negative or complex values; Spriet et al. (2025) proved that the KD representations are the only quasiprobability representations compatible with the Born rule that yield natural conditional expectations. The 2024 review by Arvidsson-Shukur et al. documents that non-positive or non-real KD values are operationally meaningful — they are resources for quantum metrology, anomalous thermodynamic work extraction, information scrambling, and violations of Leggett-Garg inequalities (temporal Bell-type constraints) and contextuality bounds (constraints on outcomes being independent of which quantities are co-measured) — and that KD cross-terms decide whether a consistent-histories description, a framework assigning probabilities to sequences of quantum events, can ascribe a classical trajectory at all.

What the modern literature has therefore settled is access and delimitation: the formal probability is measurable, uniquely characterized, and provably not a classical probability. What it has not settled is Dirac's actual question — what physical interpretation makes the complex values of this now-standard object intelligible, so that the classical-quantum analogy it anchors counts as physically satisfactory. Competing programs pursue this without consensus: Hofmann's anti-realist reading of complex probabilities as expressions of quantum determinism; operationalist readings of weak values and their critics; and the spatiotemporal program (Fullwood, Ma and Wu 2025; Fullwood and Yang 2025) that extends Dirac's construction to observables at different space-time points, where the associated 'quantum states over time' are non-Hermitian (not observable quantities themselves) and their interpretation, the constrained λn\lambda_n case, and coincident-point observables remain unresolved.

Problem Statement

Complete Dirac's classical-quantum analogy as anchored on his formal probability: give the complex values of the formal probability (the Kirkwood-Dirac distribution) for non-commuting observables a physically satisfactory interpretation. It must state what the real and imaginary parts physically mean beyond their established operational role as shifts of conjugate pointer observables, and it must account for the ordering of the observables in QKD(a,b)=Tr[ΠbρΠa]Q_{\mathrm{KD}}(a,b) = \mathrm{Tr}[\Pi_b \rho \Pi_a] rather than stipulate it: the ordering embodies Dirac's general well-ordering choice between left- and right-ordering of non-commuting observables, mirroring the ordering ambiguity in quantizing products such as qpqp, and the conjugate ordering gives the conjugate distribution with the same Terletsky-Margenau-Hill real part; time ordering is only the special case of timelike-separated correlators and trajectory weights, where it is usually assumed but not required in general, and whether the assignment of space-time labels is ambiguous or observer-dependent is an open point. The interpretation must be consistent with the established facts about the distribution — correct Born-rule marginals; accessibility of its real and imaginary parts as shifts of two conjugate pointer observables in weak-then-strong measurement; Bayes-rule behavior under changes of representation — which are post-1945 results about the same object rather than criteria Dirac himself stated, and with the known no-go and characterization constraints (no genuine joint probability with correct marginals and linearity in the state exists for non-commuting observables; the Kirkwood-Dirac representations are the only Born-compatible quasiprobability representations yielding natural conditional expectations). These published results constrain what any interpretation can achieve but do not by themselves provide one. The interpretation must recover ordinary probability theory for commuting observables and in the classical limit, including Dirac's trajectory probabilities; and it must address the scope of Dirac's program as documented in the secondary literature — well-ordered functions of non-commuting observables, the trajectory weights built from the transformation functions ⟨q′∣Q′⟩\langle q'|Q'\rangle and their generating function S=−iℏlog⁡⟨q′∣Q′⟩S = -i\hbar\log\langle q'|Q'\rangle, and the extension to constrained systems — rather than being confined to special cases; these strands delimit the general scope the interpretation must cover within one general framework.

References

  1. P. A. M. Dirac, "On the Analogy Between Classical and Quantum Mechanics", Reviews of Modern Physics 17, 195-199 (1945). DOI 10.1103/RevModPhys.17.195. https://doi.org/10.1103/RevModPhys.17.195 (Primary source; the full text is paywalled.)
  2. J. G. Kirkwood, "Quantum Statistics of Almost Classical Assemblies", Physical Review 44, 31-37 (1933). DOI 10.1103/PhysRev.44.31. https://doi.org/10.1103/PhysRev.44.31
  3. Y. Aharonov, D. Z. Albert, L. Vaidman, "How the result of a measurement of a component of the spin of a spin-1/2 particle can turn out to be 100", Physical Review Letters 60, 1351-1354 (1988). DOI 10.1103/PhysRevLett.60.1351. https://doi.org/10.1103/PhysRevLett.60.1351
  4. Charles Bamber, Jeff S. Lundeen, "Observing Dirac's classical phase space analog to the quantum state", Physical Review Letters 112, 070405 (2014). DOI 10.1103/PhysRevLett.112.070405. arXiv:1309.1491. https://doi.org/10.1103/PhysRevLett.112.070405
  5. Holger F. Hofmann, "Complex joint probabilities as expressions of reversible transformations in quantum mechanics", New Journal of Physics 14, 043031 (2012). DOI 10.1088/1367-2630-14-4-043031. arXiv:1107.0536. https://doi.org/10.1088/1367-2630-14-4-043031 (the 2011 arXiv preprint was distributed under the title "Complex joint probabilities as expressions of determinism in quantum mechanics").
  6. Holger F. Hofmann, "Reasonable conditions for joint probabilities of non-commuting observables", Quantum Studies: Mathematics and Foundations 1, 39-45 (2014). DOI 10.1007/s40509-014-0010-x. arXiv:1404.6327. https://doi.org/10.1007/s40509-014-0010-x
  7. Matteo Lostagno, Alessio Belenchia, Amikam Levy, Santiago Hernández-Gómez, Nicole Fabbri, Stefano Gherardini, "Kirkwood-Dirac quasiprobability approach to the statistics of incompatible observables", Quantum 7, 1128 (2023). DOI 10.22331/q-2023-10-09-1128. arXiv:2206.11783. https://doi.org/10.22331/q-2023-10-09-1128
  8. Stefano Gherardini, Gabriele De Chiara, "Quasiprobabilities in quantum thermodynamics and many-body systems", PRX Quantum 5, 030201 (2024). DOI 10.1103/PRXQuantum.5.030201. arXiv:2403.17138. https://doi.org/10.1103/PRXQuantum.5.030201
  9. David R. M. Arvidsson-Shukur, William F. Braasch Jr., Stephan De Bièvre, Justin Dressel, Andrew N. Jordan, Christopher Langrenez, Matteo Lostagno, Jeff S. Lundeen, Nicole Yunger Halpern, "Properties and applications of the Kirkwood-Dirac distribution", New Journal of Physics 26, 121201 (2024). DOI 10.1088/1367-2630/ada05d. arXiv:2403.18899. https://doi.org/10.1088/1367-2630/ada05d
  10. James Fullwood, Boyu Yang, "On Dirac-type correlations", arXiv:2512.08068 (2025). https://arxiv.org/abs/2512.08068
  11. James Fullwood, Zhihao Ma, Zhen Wu, "The spatiotemporal Born rule is quasiprobabilistic", arXiv:2507.16919 (2025). https://arxiv.org/abs/2507.16919
  12. Matéo Spriet, Christopher Langrenez, Raymond Brummelhuis, Stephan De Bièvre, "What is special about the Kirkwood-Dirac distributions? Only they produce natural conditional expectations", arXiv:2511.01996 (2025; posted November 2025, revised May 2026). https://arxiv.org/abs/2511.01996