Rigorous foundation and error analysis of Fermi's 1926 Gaussian-tail formula for the running maximum of partial sums of independent errors

Enrico Fermi's 1926 note 'Sopra una formula di calcolo delle probabilità' derives, by a formal passage from an exact discrete recurrence for the density of partial sums of independent, zero-mean, non-identically distributed errors — restricted to paths whose running maximum stays below a level $a$ — to the heat equation with an absorbing boundary, a closed Gaussian-tail formula for this probability that depends only on the level and the cumulative variance $t=\sum_iK_i^2$. The note defers the legitimacy of its approximations to the unstated 'conditions of the generalized Laplace formula' and provides no error bounds or validity limits. This problem asks for a complete rigorous foundation of that formula in Fermi's stated generality: precise hypotheses and a proved limit theorem justifying the discrete-to-continuum passage, and explicit quantitative error bounds in terms of the variance profile, the higher moments of the increment densities, and the ratio $a/\sqrt{t}$. The standard limit theory for random-walk maxima (invariance principles, strong approximation, and corrected diffusion approximations) covers identically distributed and Lindeberg-array increments; the quantitative treatment in Fermi's non-identically-distributed generality is not covered by it.

Background

In 1926 Enrico Fermi published a short note, 'Sopra una formula di calcolo delle probabilità' ('On a formula of probability calculus'), in Il Nuovo Cimento. The note studies sums of many small independent random errors, the classical setting of the central limit theorem, which in the older literature is called the generalized Laplace formula: if $y_1,\dots,y_n$ are independent real random variables with densities $\varphi_i$, zero means, and second moments $K_i^2=\int y^2\varphi_i(y)\,dy$, then, when each $K_i$ is small compared with the cumulative dispersion $t=\sum_{i=1}^{n}K_i^2$, the distribution of the terminal sum $\sum_i y_i$ is approximately Gaussian with variance $t$.

Fermi asked for more: the probability of a sequential (pathwise) event, namely that every partial sum simultaneously satisfies $y_1\le a$, $y_1+y_2\le a$, …, $\sum_{i=1}^{n}y_i\le a$ — equivalently, that the running maximum of the partial sums stays below the level $a$ (its complement is the probability that some partial sum exceeds $a$). His strategy, in modern language, was to introduce the sub-probability density $F(z,x)$ of the partial sum $\sum_{i\le z}y_i$ restricted to paths that have not yet crossed the level $a$ (a 'killed' random walk), write an exact discrete recurrence for $F$ (his Eq. (11)), and then formally replace the discrete difference in the index $z$ by a derivative in the cumulative variance $t=\sum_{i\le z}K_i^2$, truncating each one-step density to its second moment. This produced the heat equation $\partial F/\partial t=\frac{1}{2}\partial^2F/\partial x^2$ on the half-line $x<a$ with an absorbing boundary at $x=a$ — the forward (Fokker–Planck) equation of Brownian motion (the diffusion process that is the scaling limit of random walks, with 'time' $t$) killed at $a$. He exhibited an explicit solution built from the Gaussian heat kernel and the first-passage density $u(\tau)=\frac{a}{\sqrt{2\pi\tau^3}}e^{-a^2/2\tau}$ of Brownian motion to the level $a$ (the density of the hitting time of the level; the Lévy distribution), and integrated it to a closed Gaussian-tail formula depending only on $a$ and $t$, namely $P=\frac{2}{\sqrt{\pi}}\int_{a/\sqrt{2t}}^{\infty}e^{-x^2}\,dx$ (the complementary error function of $a/\sqrt{2t}$). The killed density, integrated over $x\le a$, yields the survival probability $2\Phi(a/\sqrt{t})-1$, where $\Phi$ is the standard normal distribution function, whereas the complementary-error-function expression is exactly the complementary crossing probability; the two conventions are related by exact complementation, so every error bound transfers verbatim and the rigorization task is identical for either convention. For equal variances $K_i=K$ this gives $t=nK^2$ and, for the crossing event, a limit of $1$ as $n\to\infty$. The note explicitly defers the analytic legitimacy of the formal steps to the conditions of legitimacy of the generalized Laplace formula and provides no error bounds. In modern terms the unproved assertions are: (a) the discrete-to-continuum passage (recurrence to absorbing heat equation) is valid when $\max_i K_i^2\ll t$ and $a\gg\max_i K_i$; (b) only the second moments of the $\varphi_i$ matter at leading order; (c) the formula remains meaningful as far as claimed.

The mathematical content sits in a well-developed neighboring field. Doob (1949) performed exactly this kind of formal passage (empirical process to Brownian bridge, i.e. Brownian motion conditioned to end at zero) to derive the Kolmogorov–Smirnov limit laws, and Donsker (1952) justified it through the invariance principle (now Donsker's theorem): partial-sum processes of independent, centered increments converge in distribution, in a function space of right-continuous paths, to Brownian motion, so that continuous functionals such as the running maximum converge as well; Prokhorov (1956) developed the general weak-convergence framework, covering independent, non-identically distributed increments under Lindeberg-type conditions (a condition ensuring no single increment dominates the total variance). Quantitative versions are furnished by the strong approximation of Komlós, Major and Tusnády (1975, 1976), who coupled partial sums with Brownian motion at optimal error rates. For the boundary-crossing probability itself, the theory of corrected diffusion approximations (Siegmund 1979; Blanchet and Glynn 2006) shows that the naive Gaussian first-passage formula for the maximum of a random walk carries a systematic, distribution-dependent error expressible through ladder-height distributions (the laws of the increments of the running maximum at the epochs where new maxima are attained), and provides corrected approximations with error control. Classical fluctuation theory (treated in Feller's Volume II) gives exact combinatorial and renewal descriptions of staying probabilities, showing that at fixed threshold with growing number of steps the leading constant depends on the full increment law, not only on the total variance.

Problem Statement

Let $y_1,\dots,y_n$ be independent real random variables with densities $\varphi_i$, zero mean, and second moments $K_i^2=\int y^2\varphi_i(y)\,dy$; let $t=\sum_{i=1}^{n}K_i^2$ and let $a>0$. Consider the probability $P_n(a)$ that the simultaneous inequalities

$$y_1\le a,\quad y_1+y_2\le a,\quad\dots,\quad\sum_{i=1}^{n}y_i\le a$$

hold, i.e. that the running maximum of the partial sums stays below $a$. (The complementary event — the probability that some partial sum exceeds $a$ — is obtained by exact complementation, so the choice of convention does not affect the task.) Fermi's derivation proceeds through the killed partial-sum density $F(z,x)$, defined as the sub-probability density at $x$ of the partial sum $\sum_{i=1}^{z}y_i$ restricted to paths whose partial sums have all stayed below $a$ ('killed' refers to the removal of paths at their first crossing of the level). Starting from an exact discrete recurrence for $F$ (his Eq. (11)), he formally replaces the discrete difference in the index $z$ by a derivative in $t=\sum_{i\le z}K_i^2$ and truncates each one-step density to its second moment, obtaining the heat equation $\partial F/\partial t=\frac{1}{2}\partial^2F/\partial x^2$ on the half-line $x<a$ with an absorbing boundary at $x=a$, whose solution he integrates to a Gaussian-tail expression depending only on $a$ and $t$. The complementary-error-function expression $\frac{2}{\sqrt{\pi}}\int_{a/\sqrt{2t}}^{\infty}e^{-x^2}\,dx=\mathrm{erfc}(a/\sqrt{2t})$ is the complementary crossing probability, while the killed density integrated over $x\le a$ yields the survival probability $2\Phi(a/\sqrt{t})-1=1-\mathrm{erfc}(a/\sqrt{2t})$, where $\Phi$ is the standard normal distribution function. Under the definition of $P_n(a)$ above, the approximation to be rigorously justified is therefore $P_n(a)\approx 2\Phi(a/\sqrt{t})-1$. The problem is to give this derivation a complete rigorous foundation in the generality in which Fermi stated it (independent, not necessarily identically distributed increments):

1. State precisely, and prove, the limit theorem that justifies the discrete-to-continuum passage: identify the exact hypotheses on the array $(\varphi_i)$, on $\max_iK_i^2/t$, and on $a$ relative to the $K_i$, under which the killed-chain recurrence converges to the absorbing heat equation and $P_n(a)$ is asymptotic to the Gaussian-tail formula $2\Phi(a/\sqrt{t})-1$. (The quantitative bounds of item 2 imply the probability-level asymptotic; the discrete-to-continuum passage is the convergence of the recurrence to the absorbing heat equation, with a density-level bound on $F$ as its quantitative form.)

2. Provide explicit, quantitative error bounds: bounds on the difference between $P_n(a)$ and the Gaussian-tail formula, and on the corresponding density-level difference for the killed density $F$ — controlling the neglected higher-moment terms of the one-step expansion — in terms of $\max_iK_i/\sqrt{t}$, the higher moments of the $\varphi_i$, and $a/\sqrt{t}$.

References

1. Enrico Fermi, 'Sopra una formula di calcolo delle probabilità', Il Nuovo Cimento 3(1–2), 313–318 (1926). DOI: 10.1007/BF02961077. https://doi.org/10.1007/BF02961077
2. J. L. Doob, 'Heuristic Approach to the Kolmogorov-Smirnov Theorems', The Annals of Mathematical Statistics 20(3), 393–403 (1949). DOI: 10.1214/aoms/1177729991. https://doi.org/10.1214/aoms/1177729991
3. Monroe D. Donsker, 'Justification and Extension of Doob's Heuristic Approach to the Kolmogorov-Smirnov Theorems', The Annals of Mathematical Statistics 23(2), 277–281 (1952). DOI: 10.1214/aoms/1177729445. https://doi.org/10.1214/aoms/1177729445
4. Yu. V. Prokhorov, 'Convergence of Random Processes and Limit Theorems in Probability Theory', Theory of Probability & Its Applications 1(2), 157–214 (1956). DOI: 10.1137/1101016. https://doi.org/10.1137/1101016
5. J. Komlós, P. Major, G. Tusnády, 'An approximation of partial sums of independent RV'-s, and the sample DF. I', Zeitschrift für Wahrscheinlichkeitstheorie und verwandte Gebiete 32, 111–131 (1975). DOI: 10.1007/BF00533093. https://doi.org/10.1007/BF00533093
6. J. Komlós, P. Major, G. Tusnády, 'An approximation of partial sums of independent RV's, and the sample DF. II', Zeitschrift für Wahrscheinlichkeitstheorie und verwandte Gebiete 34, 33–58 (1976). DOI: 10.1007/BF00532688. https://doi.org/10.1007/BF00532688
7. D. Siegmund, 'Corrected diffusion approximations in certain random walk problems', Advances in Applied Probability 11(4), 701–719 (1979). DOI: 10.2307/1426855. https://doi.org/10.2307/1426855
8. Jose Blanchet, Peter Glynn, 'Complete Corrected Diffusion Approximations for the Maximum of a Random Walk', The Annals of Applied Probability 16(2), 951–983 (2006). DOI: 10.1214/105051606000000042. https://doi.org/10.1214/105051606000000042
9. William Feller, 'An Introduction to Probability Theory and Its Applications, Volume 2', 2nd edition, Wiley. ISBN 978-0-471-25709-7. https://www.wiley.com/en-us/An+Introduction+to+Probability+Theory+and+Its+Applications%2C+Volume+2%2C+2nd+Edition-p-9780471257097