Controlled stochastic imaginary-time projection for few-fermion ground-state energies: taming the exponential variance growth

The ground-state energy of a nonrelativistic system of $N$ interacting fermions can be extracted from the large-imaginary-time behavior of many-body projection amplitudes evaluated by Monte Carlo on a space-time lattice. Alzetta, Parisi and Semeraro (1984) proposed such a scheme for the quantum few-body problem — a deterministic mean-field (saddle-point) part plus a stochastically sampled many-body correction on a coarse mesh, with variational trial states at the initial and final imaginary time — and illustrated it with preliminary numerical results for a simple lattice system of few fermions with an attractive delta interaction. The program they initiated is still incomplete: the variance of fermionic projection estimators grows exponentially with projection time, $V_N(t)\approx e^{tB_N}$ with a constant $B_N$ that increases with particle number — the central obstruction now known as the fermion noise/sign problem, proven NP-hard in its general form — and the optimization of trial states, the control of this variance growth, the extrapolation of lattice discretization errors to the continuum limit, and practical three-dimensional implementations remain open. The problem is to construct (or rigorously rule out, for a specified physically relevant class of few-fermion Hamiltonians) a stochastic projection method for ground-state energies whose total cost at fixed target accuracy is non-exponential in projection time and particle number, whose systematic lattice and constraint biases are controlled, and which is feasible for realistic three-dimensional systems with spin and isospin degrees of freedom.

Background

Nonrelativistic quantum systems containing a small number of interacting particles — electrons in light atoms and small molecules, nucleons in light nuclei, cold-atom gases — are the traditional few-body problems: the Schrödinger equation is too high-dimensional for direct diagonalization, yet the systems are too strongly correlated for mean-field methods (Hartree–Fock-type independent-particle approximations) to be quantitatively reliable. An exact-in-principle route is imaginary-time projection: the evolution operator $e^{-tH}$ at imaginary time $t$ damps every excited-state component of a trial state as $e^{-t(E_n-E_0)}$, so the ground-state energy $E_0$ can be read off from the large-$t$ behavior of the many-body propagator amplitude $\langle\Psi^{\mathrm{f}}|e^{-tH}|\Psi^{\mathrm{i}}\rangle$ between initial and final trial states (approximate wavefunctions chosen to overlap strongly with the ground state and to reduce Monte Carlo fluctuations, i.e., to act as importance-sampling weights). The amplitude is a very high-dimensional integral estimated by Monte Carlo (random sampling), with a statistical variance that shrinks only as the inverse square root of the number of samples.

The technique reached few-body fermions in the early 1980s from lattice gauge theory: Hamber and Parisi (1981) extracted hadron masses from stochastically evaluated imaginary-time correlators, and Hirsch, Scalapino, Sugar and Blankenbecler (1981) introduced the determinant Monte Carlo method, in which two-body interactions are decoupled so that the fermions integrate out to determinants that are then sampled; Negele (1982) reviewed the parallel mean-field-plus-path-integral program for nuclei. In this context Alzetta, Parisi and Semeraro (1984) proposed a new method for computing the ground-state energy of a few-body quantum system using a modified Monte Carlo technique that works for both fermionic and bosonic systems. The scheme is hybrid: the dominant mean-field contribution — obtained at the saddle point, the stationary point of the Euclidean action that gives the leading exponential behavior — is evaluated deterministically on a fine mesh, while the residual many-body correction is sampled on a much coarser space-time lattice. The method was illustrated by the preliminary numerical results of a simple lattice calculation for a system of few fermions (half spin-up, half spin-down) interacting through an attractive delta interaction (a zero-range contact potential).

The central obstruction is statistical: the variance of the fermionic projection estimator grows exponentially with imaginary time, $V_N(t)\approx e^{tB_N}$, with a constant $B_N$ that increases with particle number $N$; equivalently, the signal-to-noise ratio decays exponentially and the cost of a fixed-precision result grows exponentially with $t$ and $N$. This is the modern fermion noise problem; its extreme form is the fermion sign problem, in which the sampled weights are not all positive, importance sampling requires cancellations, and the average sign itself decays exponentially. What remains open in this program: the optimal choice of the initial and final trial states; the mitigation of the exponential variance growth; the control of lattice discretization errors and the extrapolation to the continuum (zero lattice spacing) limit; and the extension to realistic three-dimensional atomic, molecular, and nuclear systems. The practical particle-number limit of the original setup is estimated heuristically near $N\sim 10$.

Later literature addressed these items by other routes. Constrained-path Monte Carlo (Zhang, Carlson and Gubernatis, 1995) and the phase-free auxiliary-field quantum Monte Carlo of Zhang and Krakauer (2003) — in which two-body interactions are traded for fluctuating auxiliary fields and sampling is guided by Slater-determinant trial states (antisymmetrized products of single-particle orbitals) — control the sign and variance by constraining or reweighting the random walkers, at the price of a systematic constraint bias that is not a priori bounded; nuclear quantum Monte Carlo (reviewed by Carlson et al., 2015) computes light and medium-mass nuclei with fixed-node and related constraint approximations of the same type, where a fixed-node condition forbids the random walkers from crossing the nodal surface (the zero locus) of the trial wavefunction. Troyer and Wiese (2005) proved that the sign problem is NP-hard in general (a generic polynomial-time unbiased solution would solve all problems in the complexity class NP) and that the average sign decays exponentially with system size and inverse temperature; Endres, Kaplan, Lee and Nicholson (2011) showed that the noise problem appears as heavy-tailed distributions of Monte Carlo correlators, analyzed it for three-dimensional lattice fermions, and proposed alternative statistics for extracting ground-state energies. Current work continues to identify sign-problem-free classes (Wei et al., 2016, via Majorana positivity, a decomposition of certain fermion bilinear actions into nonnegative parts) and to seek general mitigations (Levy and Clark, 2019; Mondaini et al., 2022; Larson et al., 2025). The field's present state is: exact fermionic projection Monte Carlo exists but costs exponentially; polynomial-cost variants exist but carry uncontrolled bias.

Problem Statement

Given a nonrelativistic Hamiltonian $H$ for $N$ interacting fermions (two-body forces, with spin and, where relevant, isospin degrees of freedom — isospin being the internal quantum number that distinguishes protons from neutrons in nuclear systems — in one to three spatial dimensions), the ground-state energy $E_0(N)$ can be obtained from the large-imaginary-time limit of many-body projection amplitudes $\langle\Psi^{\mathrm{f}}|e^{-tH}|\Psi^{\mathrm{i}}\rangle$, evaluated stochastically on a space-time lattice. Alzetta, Parisi, and Semeraro (1984) proposed an early hybrid stochastic scheme of this kind for few-fermion ground states. The variance of such fermionic stochastic estimators grows exponentially with projection time, $V_N(t)\approx e^{tB_N}$, with a constant $B_N$ that grows with particle number, and the optimal choice of the initial/final trial states $\Psi^{\mathrm{i}}$ and $\Psi^{\mathrm{f}}$, the mitigation of this variance growth, the control of lattice discretization errors with extrapolation to the continuum limit, and practical three-dimensional implementations all remain open.

The problem is to complete this program in a controlled way: construct — or rigorously rule out for a specified, physically relevant class of few-fermion Hamiltonians — a stochastic projection method for ground-state energies that simultaneously achieves: (a) non-exponential statistical cost, meaning the total computational cost of reaching a fixed target accuracy in $E_0(N)$ grows at most polynomially with particle number and projection time (equivalently, the estimator variance does not grow exponentially with projection time $t$ and $N$, or that growth is removable at polynomial cost) without introducing uncontrolled systematic bias; (b) controlled systematic errors, meaning quantified lattice-spacing and time-step errors and a principled continuum-limit extrapolation in which stochastic and systematic error components remain simultaneously small; and (c) feasible computational cost for realistic three-dimensional few-fermion systems (atoms, molecules, or nuclei) including spin/isospin degeneracies, beyond the preliminary few-fermion lattice-test regime of the original method. Conditions (a)–(c) describe one overarching objective — a controlled, unbiased (or certifiably biased) few-fermion ground-state solver — and hold jointly: a variance control that only works with uncontrolled bias, an error control that only works where variance is already tame, or a method that only works in one dimension does not satisfy them. The class of Hamiltonians and particle numbers for which the conditions hold must be stated precisely, together with any residual bias; the claimed variance and cost scaling must be demonstrated, or, for a rule-out, it must be established that no method satisfying (a)–(c) exists for the stated class. Since the generic sign problem is known to be NP-hard (Troyer and Wiese, 2005), the specified class must be physically motivated and not trivially sign-free, with the difficulty of the problem genuinely beginning within it; symmetrically, a rule-out for that class must establish the impossibility within the class itself rather than rest on the generic NP-hardness alone.

References

1. Renzo Alzetta, Giorgio Parisi, Teresa Semeraro, On the quantum few-body problem, Nuclear Physics B 235 (1984) 576–598, DOI 10.1016/0550-3213(84)90497-8, https://doi.org/10.1016/0550-3213(84)90497-8
2. J. E. Hirsch, D. J. Scalapino, R. L. Sugar, R. Blankenbecler, Efficient Monte Carlo Procedure for Systems with Fermions, Physical Review Letters 47 (1981) 1628–1631, DOI 10.1103/PhysRevLett.47.1628, https://doi.org/10.1103/PhysRevLett.47.1628
3. J. W. Negele, The mean-field theory of nuclear structure and dynamics, Reviews of Modern Physics 54 (1982) 913–1015, DOI 10.1103/RevModPhys.54.913, https://doi.org/10.1103/RevModPhys.54.913
4. H. Hamber, G. Parisi, Numerical Estimates of Hadronic Masses in a Pure SU(3) Gauge Theory, Physical Review Letters 47 (1981) 1792–1795, DOI 10.1103/PhysRevLett.47.1792, https://doi.org/10.1103/PhysRevLett.47.1792
5. R. Alzetta, A. M. Pinna, Analytical and numerical approximations to the delta interaction, Journal of Physics A: Mathematical and General 20 (1987) 1773–1786, DOI 10.1088/0305-4470/20/7/023, https://doi.org/10.1088/0305-4470/20/7/023
6. Akira Sasaki, Masashi Kojo, Kikuji Hirose, Hidekazu Goto, Real-space finite-difference approach for multi-body systems: path-integral renormalization group method and direct energy minimization method, Journal of Physics: Condensed Matter 23 (2011) 434001, DOI 10.1088/0953-8984/23/43/434001, https://doi.org/10.1088/0953-8984/23/43/434001
7. Shiwei Zhang, J. Carlson, J. E. Gubernatis, Constrained Path Quantum Monte Carlo Method for Fermion Ground States, Physical Review Letters 74 (1995) 3652–3655, DOI 10.1103/PhysRevLett.74.3652, https://doi.org/10.1103/PhysRevLett.74.3652
8. Shiwei Zhang, Henry Krakauer, Quantum Monte Carlo Method using Phase-Free Random Walks with Slater Determinants, Physical Review Letters 90 (2003) 136401, DOI 10.1103/PhysRevLett.90.136401, https://doi.org/10.1103/PhysRevLett.90.136401
9. Matthias Troyer, Uwe-Jens Wiese, Computational Complexity and Fundamental Limitations to Fermionic Quantum Monte Carlo Simulations, Physical Review Letters 94 (2005) 170201, DOI 10.1103/PhysRevLett.94.170201, https://doi.org/10.1103/PhysRevLett.94.170201 (preprint arXiv:cond-mat/0408370, https://arxiv.org/abs/cond-mat/0408370)
10. Michael G. Endres, David B. Kaplan, Jong-Wan Lee, Amy N. Nicholson, Noise, Sign Problems, and Statistics, Physical Review Letters 107 (2011) 201601, DOI 10.1103/PhysRevLett.107.201601, https://doi.org/10.1103/PhysRevLett.107.201601 (preprint arXiv:1106.0073, https://arxiv.org/abs/1106.0073)
11. Z. C. Wei, Congjun Wu, Yi Li, Shiwei Zhang, T. Xiang, Majorana Positivity and the Fermion Sign Problem of Quantum Monte Carlo Simulations, Physical Review Letters 116 (2016) 250601, DOI 10.1103/PhysRevLett.116.250601, https://doi.org/10.1103/PhysRevLett.116.250601
12. J. Carlson, S. Gandolfi, F. Pederiva, Steven C. Pieper, R. Schiavilla, K. E. Schmidt, R. B. Wiringa, Quantum Monte Carlo methods for nuclear physics, Reviews of Modern Physics 87 (2015) 1067–1118, DOI 10.1103/RevModPhys.87.1067, https://doi.org/10.1103/RevModPhys.87.1067
13. Ryan Levy, Bryan K. Clark, Mitigating the Sign Problem Through Basis Rotations, arXiv:1907.02076 (2019), https://arxiv.org/abs/1907.02076
14. Rubem Mondaini, Sabyasachi Tarat, Richard T. Scalettar, Universality and Critical Exponents of the Fermion Sign Problem, arXiv:2207.09026 (2022), https://arxiv.org/abs/2207.09026
15. Ryan Larson, Rubem Mondaini, Richard T. Scalettar, Sign-Resolved Statistics and the Origin of Bias in Quantum Monte Carlo, arXiv:2512.04056 (2025), https://arxiv.org/abs/2512.04056