Wenzel and Wilson's 1992 Physical Review Letters paper introduced Hilbert-space basis set reduction (HS-BSR): partition a molecular-orbital basis into a small internal subset treated by (full) configuration interaction and a large external subset treated through an approximate external many-body state, thereby absorbing the dominant part of the electron-correlation cusp contribution at a cost that scales only with the second power of the number of orbitals. On the carbon atom and the beryllium dimer the method reproduced full-CI results to within 1% of the external orbitals' correlation contribution. The paper, however, left its reliability foundation open: the off-diagonal couplings between external determinants are neglected by construction with only a leading-order momentum-scaling argument for the resulting error; the choice of the internal basis is governed only by the empirical rule that the Hartree-Fock energy should be well converged on an internal double-zeta-plus-polarization subset; the four-index transformation over external indices ultimately dominates the cost; and two generalizations (localized external orbitals for large molecules, an ab initio core-valence pseudopotential) were proposed but not developed. This record asks for that missing foundation: rigorous error bounds for the neglected external-external couplings across chemical regimes, a systematically convergent internal-basis selection criterion, cost reduction of the external treatment within the BSR framework, and the feasibility and construction of the proposed core-valence pseudopotential.

Background

Accurate ab initio electronic-structure calculations must describe electron correlation, the difference between the exact many-electron energy and the Hartree-Fock (HF) mean-field energy. In practice the many-electron wave function is expanded in Slater determinants (antisymmetrized products of one-electron spin-orbitals) built from a finite basis of Gaussian functions; the basis is labeled double zeta plus polarization (DZP), triple zeta plus polarization (TZP), or quadruple zeta plus polarization (QZP) as successive shells of functions are added for each angular momentum, together with polarization functions of higher angular momentum (for example, Dunning's correlation-consistent basis sets for the atoms B through Ne). Truncating the basis makes the correlation energy converge only slowly, because the electron-electron cusp, the sharp kink the exact wave function has wherever two electrons coincide, is poorly represented by finite orbital expansions; Kutzelnigg and Klopper (1991) showed that adding terms linear in the interelectronic coordinate r12r_{12} cures this, the idea behind the later R12/F12 explicitly correlated method family (R12 and F12 are two notations for that coordinate). Meanwhile, exact diagonalization in the truncated determinant space (full configuration interaction, FCI) scales combinatorially with the number of orbitals, and even the four-index transformation, the conversion of the two-electron repulsion integrals from the atomic-orbital basis to the molecular-orbital basis, formally requires work of order N5N^5 in the number NN of basis functions and becomes a bottleneck for large bases.

Basis set reduction, proposed by Freed (1983), partitions the orbitals into a small internal set that describes the nonperturbative single-particle structure (the gross charge distribution of the atom or molecule) and a large external set that carries the cusp-related correlation; an effective Hamiltonian acting only on the internal space then accounts for the external orbitals perturbatively. The traditional obstacle is that this effective Hamiltonian must contain three-electron terms, which already arise in first order of perturbation theory and scale with the third power of the number of external orbitals. The Hilbert-space variant, HS-BSR, of Wenzel and Wilson (1992) never constructs an effective Hamiltonian. It computes an internal many-body state by a configuration-interaction (CI) calculation, selects determinants whose coefficients exceed a threshold as root determinants, generates their single and double excitations into the external space, solves a small eigenvalue problem for each such external block retaining its dominant coupling to the internal state, and finally minimizes the internal energy in the effective field generated by the external many-body state. The essential approximation is the neglect of off-diagonal matrix elements between different external (mainly doubly external) determinants; the most important three-electron contributions are absorbed implicitly without ever being computed; correspondingly, integrals with more than two external indices do not appear in the calculation. On the 3^3P state of the carbon atom (Dunning DZP, TZP, and QZP bases) the basis-set-reduction error saturated below 1% of the external orbitals' correlation contribution, and for the beryllium dimer, a hard test because Be2\mathrm{Be}_2 is unbound at the HF level and its shallow well stems entirely from correlation (Harrison and Handy's FCI benchmarks), the method reproduced the FCI equilibrium distance within about 0.05 a.u. The external step scales only as the square of the number of external orbitals and parallelizes well. A momentum-space scaling argument explains the accuracy: the basis-set truncation error of second-order perturbation theory falls with the second inverse power of the maximal basis momentum, the intrinsic error of second-order perturbation theory with the third (Wenzel, Steiner, and Wilson, 1998), and couplings between two external configurations appear first in second order, suppressed by the momentum separation between internal (valence) and external orbitals.

What the 1992 paper did not supply is any control over its approximations. The internal basis must be chosen so that the HF energy is well converged on it, stated as the requirement that triple excitations into the external space remain unimportant; the only guidance is the empirical rule that an internal DZP usually suffices, and the paper documents the failure mode: reducing a TZP basis to a minimal internal basis inflates the error to 66% of the external contribution. The neglect of external-external off-diagonal couplings is justified only by the leading-order scaling estimate, not by any bound. The four-index transformation over the external indices ultimately dominates the computational effort; the authors note that the external orbitals need not be HF orbitals, so they could be localized using standard localization techniques (Sæbø and Pulay, 1985) to exploit spatial locality in large molecules, and they speculate that the same formalism could generate an ab initio pseudopotential for the core electrons of heavy elements to treat core-valence correlation. None of these directions was developed in the source, and constructing any such effective core pseudopotential would face the transferability problem that pseudopotential design has always found difficult.

Problem Statement

Make Hilbert-space basis set reduction (HS-BSR) as introduced by Wenzel and Wilson (1992) a certifiably reliable and scalable method, by resolving the issues its authors left open. The common objective binding the components is that the accuracy of the internal/external partition become predictable and certifiable rather than empirical; the components jointly determine whether the method's demonstrated cost advantage can be used with guaranteed accuracy.

(1) Error control: establish rigorous, quantified bounds (a priori or a posteriori) on the error introduced by neglecting the off-diagonal many-body matrix elements between external determinants, valid across chemical regimes including weakly correlated, multireference, and near-degenerate situations, going beyond the leading-order momentum-scaling estimate; include the combined effect of the truncation of the internal configuration-interaction treatment. The quantity to be bounded is the source's basis-set-reduction error: the difference between the full-CI valence energy of the full basis and the energy obtained from basis set reduction, which the source measures against the external orbitals' correlation contribution.

(2) Internal-basis selection: formulate a systematically convergent criterion for choosing the internal subset, replacing the empirical rule that an internal double-zeta-plus-polarization basis usually suffices for the Hartree-Fock energy to be well converged on the internal basis (with the documented failure modes, for example the 66% error of the external contribution when a TZP basis is reduced to a minimal internal basis).

(3) Cost and scalability: reduce or distribute the four-index transformation over the external indices in a manner compatible with the BSR construction (in particular preserving the property that matrix elements involving more than two external indices never appear), so that the cost advantage extends to spatially extended and large molecules.

(4) Proposed generalization: determine the feasibility, accuracy, and algorithmic construction of an ab initio pseudopotential for the core electrons of heavy elements derived from the BSR formalism for the treatment of core-valence correlation, the generalization the source proposed without developing it.

References

  1. Wolfgang Wenzel and Kenneth G. Wilson, 'Basis set reduction in Hilbert space', Physical Review Letters 69, 800-803 (1992), DOI: 10.1103/PhysRevLett.69.800, https://doi.org/10.1103/PhysRevLett.69.800
  2. Wolfgang Wenzel, Matthew M. Steiner, and Kenneth G. Wilson, 'Multireference basis-set reduction', International Journal of Quantum Chemistry 60, 1325-1330 (1996), DOI: 10.1002/(SICI)1097-461X(1996)60:7<1325::AID-QUA14>3.0.CO;2-3, https://doi.org/10.1002/(SICI)1097-461X(1996)60:7%3C1325::AID-QUA14%3E3.0.CO;2-3
  3. W. Wenzel, M. M. Steiner, and K. G. Wilson, 'Scaling behavior of dynamic correlation effects', International Journal of Quantum Chemistry 70, 147-157 (1998), DOI: 10.1002/(SICI)1097-461X(1998)70:1<147::AID-QUA12>3.0.CO;2-C, https://doi.org/10.1002/(SICI)1097-461X(1998)70:1%3C147::AID-QUA12%3E3.0.CO;2-C
  4. Wolfgang Wenzel and Matthew M. Steiner, 'Brillouin-Wigner based multi-reference perturbation theory for electronic correlation effects', The Journal of Chemical Physics 108, 4714-4724 (1998), DOI: 10.1063/1.475921, https://doi.org/10.1063/1.475921
  5. P. Stampfuß and W. Wenzel, 'Improved implementation and application of the individually selecting configuration interaction method', The Journal of Chemical Physics 122, 024110 (2005), DOI: 10.1063/1.1829045, https://doi.org/10.1063/1.1829045
  6. Yibo Lei, Yubin Wang, Huixian Han, Qi Song, Bingbing Suo, and Zhenyi Wen, 'New implementation of the configuration-based multi-reference second order perturbation theory', The Journal of Chemical Physics 137, 144102 (2012), DOI: 10.1063/1.4757264, https://doi.org/10.1063/1.4757264
  7. Karl F. Freed, 'Is there a bridge between ab initio and semiempirical theories of valence?', Accounts of Chemical Research 16, 137-144 (1983), DOI: 10.1021/ar00088a004, https://doi.org/10.1021/ar00088a004
  8. Werner Kutzelnigg and Wim Klopper, 'Wave functions with terms linear in the interelectronic coordinates to take care of the correlation cusp. I. General theory', The Journal of Chemical Physics 94, 1985-2001 (1991), DOI: 10.1063/1.459921, https://doi.org/10.1063/1.459921
  9. Thom H. Dunning, 'Gaussian basis sets for use in correlated molecular calculations. I. The atoms boron through neon and hydrogen', The Journal of Chemical Physics 90, 1007-1023 (1989), DOI: 10.1063/1.456153, https://doi.org/10.1063/1.456153
  10. Svein Sæbø and Peter Pulay, 'Local configuration interaction: An efficient approach for larger molecules', Chemical Physics Letters 113, 13-18 (1985), DOI: 10.1016/0009-2614(85)85003-X, https://doi.org/10.1016/0009-2614(85)85003-X
  11. R. J. Harrison and N. C. Handy, 'Full CI results for Be2 and (H2)2 in large basis sets', Chemical Physics Letters 98, 97-101 (1983), DOI: 10.1016/0009-2614(83)87105-X, https://doi.org/10.1016/0009-2614(83)87105-X
  12. Christof Hättig, Wim Klopper, Andreas Köhn, and David P. Tew, 'Explicitly Correlated Electrons in Molecules', Chemical Reviews 112, 4-74 (2012), DOI: 10.1021/cr200168z, https://doi.org/10.1021/cr200168z
  13. Christoph Riplinger and Frank Neese, 'An efficient and near linear scaling pair natural orbital based local coupled cluster method', The Journal of Chemical Physics 138, 034106 (2013), DOI: 10.1063/1.4773581, https://doi.org/10.1063/1.4773581
  14. Sandeep Sharma, Adam A. Holmes, Guillaume Jeanmairet, Ali Alavi, and C. J. Umrigar, 'Semistochastic Heat-Bath Configuration Interaction Method: Selected Configuration Interaction with Semistochastic Perturbation Theory', Journal of Chemical Theory and Computation 13, 1595-1604 (2017), DOI: 10.1021/acs.jctc.6b01028, https://doi.org/10.1021/acs.jctc.6b01028