Anderson's self-consistent pseudopotential program for energy bands: optimal outer Hamiltonian, controlled localization tails, and a weak-perturbative regime for all bands of interest

In 1968, P. W. Anderson proposed a self-consistent pseudopotential formalism for energy bands that generates nonorthogonal "ultralocalized" functions — more localized than Wannier functions — as exact solutions of a pseudopotential equation involving a freely chosen "outer" Hamiltonian $\mathcal{H}^{\mathrm{out}}$, and speculated that the pseudopotential could be made weak enough that low-order perturbation theory would solve the one-electron problem for all bands of interest, including $d$ bands and covalent bond bands. The paper left the program open on five fronts: no criterion for an optimal $\mathcal{H}^{\mathrm{out}}$, unquantified exponential tails, no numerical demonstration, no extension to non-periodic or interacting systems, and no proof of the weak-perturbative regime. The direct lineage — Anderson's own $\pi$-electron demonstration (1969), the Weeks–Anderson–Davidson non-Hermitian development (1973), localized-orbital band-structure applications with mixed success (Kunz; Mašović), and the Foulkes–Edwards Kronig–Penney analysis proving that the original equation has no solutions for weak potentials and that its bases are never unique — together with the adjacent frameworks that absorbed parts of the agenda (maximally localized Wannier functions, decay and localization theory, non-orthogonal localized orbitals in silicon, and modern nonempirical tight-binding) leave a precise open core: no construction within the program supplies a principled (unique or optimal) $\mathcal{H}^{\mathrm{out}}$, and no demonstration establishes the weak-perturbative regime for $d$-band or covalent-band materials.

Background

Band theory describes the one-electron states of a perfect crystal as Bloch functions: delocalized eigenfunctions of the form $\psi_k(r)=e^{ik\cdot r}u_k(r)$, labeled by a crystal momentum $k$ in the Brillouin zone (the primitive cell of the crystal's reciprocal lattice). Wannier showed in 1937 that the same information can be carried by localized functions, now called Wannier functions: the Fourier transform of the Bloch functions over the Brillouin zone, $W_n(r)=N^{-1}\sum_k e^{ik\cdot R_n}\psi_k(r)$, centered at lattice sites $R_n$, with a freely chosen phase ("gauge") for each $\psi_k$. For an isolated band the Wannier functions can be made orthonormal and exponentially localized, and classical analyticity results (Kohn 1959, and related work summarized in Anderson's paper) show that the asymptotic decay rate is set by branch points — points in complex crystal momentum at which the given band meets a neighboring band — so the tail is governed by the smallest relevant band gap. Orthonormality is nevertheless expensive in practice: the most localized Wannier functions of a partially filled or narrow band carry large negative amplitudes on neighboring sites, a nonlocality that is a consequence of forced orthogonality rather than of the band's physics.

The pseudopotential idea, developed for the core-valence problem in the 1950s and consolidated by Austin, Heine, and Sham, removes exactly this kind of orthogonality artifact: inside an atomic core, the strong attractive potential and the repulsive effect of orthogonalization to the core states largely cancel, leaving a weak effective operator for the valence electron. Anderson's 1968 Letter imported this logic from the core-valence problem to the problem of inter-site localization of band functions. He first observed, by an explicit calculation of the mean-square radius, that a normalized nonorthogonal combination $\varphi_0\propto W_0+\sum_{n\neq 0}S_nW_n$ of Wannier functions can always be made more localized than $W_0$ itself (the optimal mixing coefficients are positive for an $s$ band, making $\varphi_n$ more atomic-orbital-like). He then derived the wave equation that such ultralocalized functions satisfy exactly. The general form is a pseudopotential equation

$$\mathcal{H}\varphi_n = E\varphi_n + \sum_{m\neq n}(m|\mathcal{H}^{\mathrm{out}}|n)\,\varphi_m ,$$

where $\mathcal{H}$ is the true Hamiltonian of the periodic system, $E$ the band energy, and $\mathcal{H}^{\mathrm{out}}$ — the outer Hamiltonian — any operator whose matrix elements between functions centered at $R_m$ and $R_n$ depend only on $R_m-R_n$. Any localized solution generates, by Bloch summation, exact eigenfunctions of $\mathcal{H}$, with the exact band energies $E(k)$ given by matrix elements alone; the energy expression involves no overlap-matrix elements even though the functions are nonorthogonal, the price being that the pseudo-Hamiltonian is non-Hermitian. Anderson proposed a concrete $\mathcal{H}^{\mathrm{out}}$ (his Eqs. 12–13) constructed so that the pseudo-Hamiltonian acting on the function at site $n$ is essentially the kinetic energy plus the site's own atomic potential, $T+V_n$, plus a subtracted combination of the other centers' potentials from which most of the binding strength has cancelled. Then $\varphi_n$ is approximately an atomic-like eigenfunction of $T+V_n$, the residual pseudopotential is, in his words, "weak and can be treated by low-order perturbation theory", and self-consistency (the pseudopotential depends on the very $\varphi_n$ it determines) is easily achieved; as a consistency check he noted that the overlap matrix then satisfies $S\approx H/E$.

The formalism was presented against the background of a molecular program in which Adams, and Edmiston and Ruedenberg, had introduced self-consistent pseudopotentials within Hartree-Fock theory (the standard independent-electron mean-field approximation in which each electron moves in the average field of all the others) to construct localized molecular orbitals. Anderson's critique of that school was that its Hermitian pseudo-Hamiltonian, built with the density-matrix projector, reintroduces a nonlocality as severe as the Wannier tails, restricts meaningful localization to full bands (closed shells and saturated bonds), and blocks the use of perturbation theory.

The Letter is four pages of formalism with no numerical demonstration, and it explicitly leaves its program open on five fronts, which frame the open agenda taken up here: no method for choosing a best $\mathcal{H}^{\mathrm{out}}$; exponential tails of the $\varphi_n$ that are branch-point-controlled and possibly small but neither eliminated nor quantified; no demonstration on real band structures; no concrete extension to non-periodic systems or to interaction effects (only the statement that "periodicity is not at all essential", and critique rather than development of the Hartree-Fock route); and the conjecture that the pseudopotential can be made weak enough for low-order perturbation theory for all bands of interest, including $d$ bands (bands derived from atomic $d$ orbitals, as in transition metals) and covalent bond bands (the bonding valence bands of covalently bonded solids, such as silicon, diamond, and III–V semiconductors), stated without proof or example. The Letter is the acknowledged origin of the "chemical pseudopotential" method and an ancestor of modern localized-orbital methodology; its unfinished agenda is the subject of this problem.

Problem Statement

The problem is to complete Anderson's self-consistent pseudopotential program for energy bands, addressing the open fronts that the source paper's own stated limitations identify. These fronts jointly realize one scientific objective: a quantitative theory of energy bands built on ultralocalized (nonorthogonal) functions and a weak self-consistent pseudopotential.

(1) Principled outer Hamiltonian. Working with the pseudopotential equation

$$\mathcal{H}\varphi_n = E\varphi_n + \sum_{m\neq n}(m|\mathcal{H}^{\mathrm{out}}|n)\,\varphi_m$$

determine whether it admits a concrete $\mathcal{H}^{\mathrm{out}}$ (or a generalization of it, if the original form is deficient), together with a criterion — variational, topological, or otherwise — that makes this choice canonical or optimal, or unique within a class of admissible outer Hamiltonians that is specified independently of the proposed choice (uniqueness must not be obtained by restricting attention to a class defined so as to contain only the proposed operator), thereby resolving the source's admission of having no method for choosing a best outer Hamiltonian. The known generic non-uniqueness of chemical-pseudopotential bases must be confronted, not ignored. Either such a construction and criterion exist, or no admissible criterion makes the choice canonical or optimal.

(2) Localization and tails. Prove or measure that the resulting functions are more localized than the corresponding maximally localized Wannier functions of the same band, measured by the mean-square radius used in the source's own argument, and establish quantitatively the exponential tails (branch-point-controlled) — in particular whether they are “rather small quantitatively” in real applications. Either the resulting functions are more localized with quantitatively established tails, or no admissible choice attains localization better than the maximally localized Wannier functions.

(3) Demonstration on hard bands. Determine, by numerical study of real band structures beyond the author's delivered Hückel/$\pi$-electron application, the outcome for the hard cases the program targets — $d$ bands and covalent bond bands — the cases the source speculates about but does not treat. For each of the two hard classes, $d$ bands and covalent bond bands, the question is whether the method succeeds or fails on that class.

(4) Scope beyond the periodic one-electron problem. The source asserts that periodicity is not essential to the construction but gives no recipe for non-periodic or disordered systems, and critiques rather than develops the Hartree-Fock route to interaction effects; extending the construction beyond the periodic one-electron problem therefore remains an open direction of the program. This front comprises three separate extensions of the construction — to non-periodic systems, to disordered systems, and to interaction effects such as Hartree-Fock exchange and correlation — for each of which the question is whether the construction extends to that case.

(5) The weak-perturbative regime. Settle whether the self-consistent pseudopotential can be made weak enough that low-order perturbation theory suffices for all bands of interest, including $d$ bands and covalent bond bands, or whether there is a class of band structures for which no admissible outer Hamiltonian yields a perturbatively weak pseudopotential. Operationally, "low-order" means a finite order of the perturbation expansion, at most second order, stated explicitly by the construction; "suffices" means that the expansion truncated at that order reproduces the band energies of the bands of interest to a stated accuracy; and "weak enough" means that the construction exhibits its expansion parameter together with an explicit bound on it or the regime in which it is small — only the existence of such a controllable small parameter is required, not a particular value. The expansion parameter, its control, and the accuracy target are part of the construction to be supplied rather than fixed in advance.

These fronts are not independent questions: the choice in (1) determines the small parameter whose control is asked for in (5), which in turn is what makes the demonstrations in (3) and the extensions in (4) computationally and physically meaningful, while (2) certifies the localization that motivates the whole construction.

References

1. Gregory H. Wannier, “The Structure of Electronic Excitation Levels in Insulating Crystals”, Physical Review 52, 191 (1937). DOI: 10.1103/PhysRev.52.191. https://doi.org/10.1103/PhysRev.52.191
2. W. Kohn, “Analytic Properties of Bloch Waves and Wannier Functions”, Physical Review 115, 809 (1959). DOI: 10.1103/PhysRev.115.809. https://doi.org/10.1103/PhysRev.115.809
3. W. H. Adams, “On the Solution of the Hartree-Fock Equation in Terms of Localized Orbitals”, The Journal of Chemical Physics 34, 89–102 (1961). DOI: 10.1063/1.1731622. https://doi.org/10.1063/1.1731622
4. B. J. Austin, V. Heine, and L. J. Sham, “General Theory of Pseudopotentials”, Physical Review 127, 276 (1962). DOI: 10.1103/PhysRev.127.276. https://doi.org/10.1103/PhysRev.127.276
5. C. Edmiston and K. Ruedenberg, “Localized Atomic and Molecular Orbitals”, Reviews of Modern Physics 35, 457 (1963). DOI: 10.1103/RevModPhys.35.457. https://doi.org/10.1103/RevModPhys.35.457
6. Philip W. Anderson, “Self-Consistent Pseudopotentials and Ultralocalized Functions for Energy Bands”, Physical Review Letters 21, 13 (1968). DOI: 10.1103/PhysRevLett.21.13. https://doi.org/10.1103/PhysRevLett.21.13
7. Philip W. Anderson, “Localized Orbitals for Molecular Quantum Theory. I. The Hückel Theory”, Physical Review 181, 25 (1969). DOI: 10.1103/PhysRev.181.25. https://doi.org/10.1103/PhysRev.181.25
8. A. Barry Kunz, “Localized Orbitals in Polyatomic Systems”, Physica Status Solidi B 36, 301 (1969). DOI: 10.1002/PSSB.19690360131. https://doi.org/10.1002/PSSB.19690360131
9. V. Heine and D. Weaire, “Pseudopotential Theory of Cohesion and Structure”, Solid State Physics 24, 249–463 (1970). DOI: 10.1016/S0081-1947(08)60071-5. https://doi.org/10.1016/S0081-1947(08)60071-5
10. J. D. Weeks, P. W. Anderson, and A. G. H. Davidson, “Non-Hermitian Representations in Localized Orbital Theories”, The Journal of Chemical Physics 58, 1388 (1973). DOI: 10.1063/1.1679371. https://doi.org/10.1063/1.1679371
11. P. W. Anderson and J. D. Weeks, “Developments in Localized Pseudopotential Methods”, in Computational Methods for Large Molecules and Localized States in Solids (IBM Research Symposia Series), pp. 251–260 (1973). DOI: 10.1007/978-1-4684-2013-5_22. https://doi.org/10.1007/978-1-4684-2013-5_22
12. D. W. Bullett, “The Renaissance and Quantitative Development of the Tight-Binding Method”, Solid State Physics 35, 129–214 (1980). DOI: 10.1016/S0081-1947(08)60504-4. https://doi.org/10.1016/S0081-1947(08)60504-4
13. A. P. Sutton, M. W. Finnis, D. G. Pettifor, and Y. Ohta, “The Tight-Binding Bond Model”, Journal of Physics C: Solid State Physics 21, 35 (1988). DOI: 10.1088/0022-3719/21/1/007. https://doi.org/10.1088/0022-3719/21/1/007
14. D. R. Mašović, “Application of the Chemical Pseudopotential Method to the Calculation of the Energy Band Structure of CuI V”, Journal of Physics: Condensed Matter 1, 7315 (1989). DOI: 10.1088/0953-8984/1/40/007. https://doi.org/10.1088/0953-8984/1/40/007
15. W. M. C. Foulkes and D. M. Edwards, “Perfect Localized Basis Functions for Solids: Chemical Pseudopotentials and the Kronig-Penney Model”, Journal of Physics: Condensed Matter 5, 7987 (1993). DOI: 10.1088/0953-8984/5/43/013. https://doi.org/10.1088/0953-8984/5/43/013
16. Nicola Marzari and David Vanderbilt, “Maximally Localized Generalized Wannier Functions for Composite Energy Bands”, Physical Review B 56, 12847 (1997). DOI: 10.1103/PhysRevB.56.12847. https://doi.org/10.1103/PhysRevB.56.12847
17. Jens Jørgen Mortensen and Michele Parrinello, “Localized Non-Orthogonal Orbitals in Silicon”, Journal of Physics: Condensed Matter 13, 5731 (2001). DOI: 10.1088/0953-8984/13/25/301. https://doi.org/10.1088/0953-8984/13/25/301
18. Lixin He and David Vanderbilt, “Exponential Decay Properties of Wannier Functions and Related Quantities”, Physical Review Letters 86, 5341 (2001). DOI: 10.1103/PhysRevLett.86.5341. https://doi.org/10.1103/PhysRevLett.86.5341
19. Denis Usvyat and Martin Schütz, “Nonorthogonal Ultralocalized Functions and Fitted Wannier Functions for Local Electron Correlation Methods for Solids”, Theoretical Chemistry Accounts 114, 276 (2005). DOI: 10.1007/s00214-005-0685-5. https://doi.org/10.1007/s00214-005-0685-5
20. Christian Brouder, Gianluca Panati, Matteo Calandra, Christophe Mourougane, and Nicola Marzari, “Exponential Localization of Wannier Functions in Insulators”, Physical Review Letters 98, 046402 (2007). DOI: 10.1103/PhysRevLett.98.046402. https://doi.org/10.1103/PhysRevLett.98.046402
21. Nicola Marzari, Arash A. Mostofi, Jonathan R. Yates, Ivo Souza, and David Vanderbilt, “Maximally Localized Wannier Functions: Theory and Applications”, Reviews of Modern Physics 84, 1419 (2012). DOI: 10.1103/RevModPhys.84.1419. https://doi.org/10.1103/RevModPhys.84.1419
22. Aaron C. West, Michael W. Schmidt, Mark S. Gordon, and Klaus Ruedenberg, “A Comprehensive Analysis of Molecule-Intrinsic Quasi-Atomic, Bonding, and Correlating Orbitals. I. Hartree-Fock Wave Functions”, The Journal of Chemical Physics 139, 234107 (2013). DOI: 10.1063/1.4840776. https://doi.org/10.1063/1.4840776
23. Alexander V. Mironenko, Lanie Leung, and Jiqing Zhuang, “Self-Consistent Equations for Nonempirical Tight-Binding Theory”, The Journal of Chemical Physics 163, 164119 (2025). DOI: 10.1063/5.0276043. https://doi.org/10.1063/5.0276043