Valid and tight finite-sample confidence intervals for rare-event Bernoulli proportions in fixed-sample-size and targeted-stopping settings

Naive Monte Carlo estimation of a rare-event probability $p$ uses the sample proportion $\hat p$ of successes in $n$ Bernoulli trials (or, under targeted stopping, sampling until $n_0$ successes with $\hat p=n_0/N$, $N$ the random total number of trials). Existing confidence intervals force a choice: the CLT and Wilson intervals have analyzable half-widths of order $\hat p/\sqrt{\hat s}$, $\hat s=n\hat p$ the observed success count, but no finite-sample coverage guarantee; the exact (Clopper–Pearson) interval is valid for all $n$ and $p$ but conservative and admits no analytic half-width characterization; and the Chernoff and Berry–Esseen intervals derived by Bai and Lam (2025) are valid and explicit but extremely conservative, with a trivial lower bound in the fixed-$n$ setting and trivial bounds for small success counts under targeted stopping. The open problem is to construct intervals that are simultaneously finite-sample valid, substantially less conservative in the rare-event regime (non-trivial at small positive success counts and small $n_0$), and analytically transparent with provable half-width scaling.

Background

Rare-event estimation asks for the probability $p=\mathbb{P}(A)$ of an event that is very unlikely but consequential — for example a failure scenario in autonomous-vehicle safety evaluation, a system failure in reliability engineering, or a ruin event in insurance. When $A$ can be simulated but the underlying model is too complex for efficiency-guaranteed variance reduction techniques such as importance sampling or multilevel splitting, the fallback is naive Monte Carlo: run $n$ independent trials and record the indicators $I_1,\dots,I_n$ with $\mathbb{P}(I_i=1)=p$, so the indicators are Bernoulli($p$) and the estimator of $p$ is the sample proportion $\hat p=\hat s/n$, where $\hat s=\sum_{i=1}^n I_i$ is the observed number of rare-event hits. Uncertainty is quantified by a confidence interval (CI) at nominal level $1-\alpha$, and two properties are wanted simultaneously. Validity means the coverage probability $\mathbb{P}(p\in[\hat p_l(\alpha),\hat p_u(\alpha)])$ is at least $1-\alpha$ for every $p$ and every sample size — with no asymptotic caveats. Tightness is measured by Bai and Lam through the half-width, i.e. the upper gap $\hat p_u(\alpha)-\hat p$ and the lower gap $\hat p-\hat p_l(\alpha)$, with smaller gaps and explicit formulas for them both preferred.

The classical menu for a binomial proportion has a fundamental gap. The CLT interval $\hat p\pm z_{1-\alpha/2}\sqrt{\hat p(1-\hat p)/n}$ and Wilson's interval (Wilson 1927, which solves the normal-approximation inversion using the exact binomial variance structure) both have half-width of order $\hat p/\sqrt{\hat s}$, but neither is valid in the finite-sample sense: when $np$ is small their coverage can fall well below the nominal level. The exact or Clopper–Pearson interval (Clopper and Pearson 1934), obtained by inverting binomial tail probabilities, is valid for all $n$ and $p$ but is conservative (coverage strictly above nominal) and its endpoints are implicit beta-quantile solutions with no elementary closed form, so its half-width is, in the words of Bai and Lam, 'hard to analyze'. Bai and Lam (2025, Journal of Applied Probability 62) added two intervals by relaxing the exact confidence region through two classical probability bounds: Chernoff's inequality (a multiplicative exponential tail bound for binomial sums) and the Berry–Esseen theorem (the finite-sample refinement of the central limit theorem, which bounds the gap between the distribution of the standardized sum $(\hat p-p)\sqrt{n/(p(1-p))}$ and the standard normal distribution function $\Phi$ by $C/\sqrt{np(1-p)}$ for a universal constant $C$). Both new intervals are valid and explicit enough to analyze — their analysis shows, for example, that the CLT upper bound stays within $O(\hat p/\hat s)$ of a valid upper bound — but both are extremely conservative: the Berry–Esseen interval yields only a trivial (zero) lower confidence bound in the fixed-$n$ setting, and its construction assumes $p<1/2$ known a priori. The paper also analyzes a second, natural sampling scheme, targeted stopping: sampling continues until a preset number $n_0$ of successes is observed, so the inter-success gaps are geometric($p$), the total trial count $N$ is negative-binomial (the distribution of the number of trials needed to reach a fixed number of successes), and $\hat p=n_0/N$. There the Berry–Esseen interval is trivial for small $n_0$ and the Chernoff interval is extremely conservative. The paper's own recommendation is pragmatic — use the exact interval when a guarantee is required, Wilson's otherwise — which concedes that no interval is simultaneously valid, tight in the rare-event regime, and analytically transparent.

One flagged route toward a better tradeoff uses studentized (also called self-normalized) statistics: replace the unknown standard deviation $\sqrt{p(1-p)/n}$ in the standardized proportion by its empirical counterpart $\sqrt{\hat p(1-\hat p)/n}$. Berry–Esseen-type bounds for such statistics are an established topic in probability — an exponential nonuniform bound for self-normalized sums (Jing and Wang 1999), the exact convergence rate and leading term for Student's $t$ statistic (Hall and Wang 2004), and relative-error central limit theorems for the $t$ statistic with applications (Wang and Hall 2009) — and Bai and Lam cite Jing and Wang (1999) and Wang and Hall (2009) from this literature while leaving their use for proportion CIs as 'a potential future investigation'. Separately, an analytic characterization of the exact interval's half-width scaling as $p\to0$ (with $\hat s=O(1)$) would supply the missing benchmark against which the tightness of any valid construction is judged; existing length analyses of the Clopper–Pearson interval (Thulin 2014) operate in the fixed-$p$, large-$n$ regime instead.

Problem Statement

Construct confidence intervals for a rare Bernoulli probability $p$ from the sample proportion, in the fixed-sample-size setting ($n$ i.i.d. Bernoulli($p$) trials, $\hat p=\hat s/n$) and in the targeted-stopping setting (sampling until a preset number $n_0$ of successes, $\hat p=n_0/N$ with $N$ negative-binomial), that simultaneously achieve:

1. Finite-sample validity: coverage at least $1-\alpha$ for every $p$ and every $n$ (respectively every $n_0$), ideally without the prior assumption $p<1/2$ that the Berry–Esseen construction of Bai and Lam needs.
2. Rare-event tightness with provable scaling: closed-form or sharp asymptotic half-width bounds in the rare-event regime ($\hat s=n\hat p$ small, $p\to0$) that are substantially less conservative than the Chernoff and non-studentized Berry–Esseen intervals of Bai and Lam — in particular a non-trivial lower confidence bound at small positive success counts (including $\hat s=1$; at $\hat s=0$, validity for every $p$ forces the lower endpoint to be $0$, as for the exact interval), and, in the targeted-stopping setting, intervals that remain non-trivial and analyzable for small $n_0$, where their Berry–Esseen interval is trivial and their Chernoff interval extremely conservative.
3. Analytic transparency: explicit endpoint formulas or asymptotically characterizable half-widths.

References

1. Yuanlu Bai and Henry Lam (2025), 'Uncertainty quantification and confidence intervals for naive rare-event estimators', Journal of Applied Probability 62(1), 84–110. DOI: 10.1017/jpr.2024.43, https://doi.org/10.1017/jpr.2024.43 (preprint arXiv:2305.02434, https://arxiv.org/abs/2305.02434)
2. C. J. Clopper and E. S. Pearson (1934), 'The use of confidence or fiducial limits illustrated in the case of the binomial', Biometrika 26(4), 404–413. DOI: 10.1093/biomet/26.4.404, https://doi.org/10.1093/biomet/26.4.404
3. E. B. Wilson (1927), 'Probable inference, the law of succession, and statistical inference', Journal of the American Statistical Association 22(158), 209–212. DOI: 10.1080/01621459.1927.10502953, https://doi.org/10.1080/01621459.1927.10502953
4. Bing-Yi Jing and Qiying Wang (1999), 'An exponential nonuniform Berry–Esseen bound for self-normalized sums', The Annals of Probability 27(4), 2068–2088. DOI: 10.1214/aop/1022874829, https://doi.org/10.1214/aop/1022874829
5. Peter Hall and Qiying Wang (2004), 'Exact convergence rate and leading term in central limit theorem for student's t statistic', The Annals of Probability 32(2), 1419–1437. DOI: 10.1214/009117904000000252, https://doi.org/10.1214/009117904000000252
6. Qiying Wang and Peter Hall (2009), 'Relative errors in central limit theorems for student's t statistic, with applications', Statistica Sinica 19(1), 343–354. No Crossref DOI registered. https://www3.stat.sinica.edu.tw/statistica/j19n1/J19N119/J19N119.html
7. Måns Thulin (2014), 'The cost of using exact confidence intervals for a binomial proportion', Electronic Journal of Statistics 8(1). DOI: 10.1214/14-EJS909, https://doi.org/10.1214/14-EJS909
8. Owen McGrath and Kevin Burke (2024), 'Binomial confidence intervals for rare events: importance of defining margin of error relative to magnitude of proportion', The American Statistician 78(4), 437–449. DOI: 10.1080/00031305.2024.2350445, https://doi.org/10.1080/00031305.2024.2350445
9. Yuan-Lu Bai, Zhi-Yuan Huang, Henry Lam and Ding Zhao (2025), 'Black-box rare-event simulation for safety testing of AI agents: an overview', Journal of the Operations Research Society of China 13, 750–774. DOI: 10.1007/s40305-025-00585-0, https://doi.org/10.1007/s40305-025-00585-0
10. Morgane Austern and Lester Mackey (2022), 'Efficient concentration with Gaussian approximation', arXiv:2208.09922, https://arxiv.org/abs/2208.09922
11. Jiahao Ming, Aaditya Ramdas, Yi Shen, Ruodu Wang and Ian Waudby-Smith (2026), 'Gaffke's confidence interval for the mean of bounded data is inadmissible but asymptotically efficient', arXiv:2607.18661, https://arxiv.org/abs/2607.18661
12. Aiyou Chen, Ruixuan Rachel Zhou, Joseph J. Lee, Nicholas Chamandy and Henning Hohnhold (2026), 'Confidence intervals for rate estimation with importance sampling in autonomous vehicle evaluation', arXiv:2604.03827, https://arxiv.org/abs/2604.03827
13. Minjun Cho, Yuetian Luo and Chao Gao (2025), 'Robust confidence intervals for a binomial proportion: local optimality and adaptivity', arXiv:2509.05568, https://arxiv.org/abs/2509.05568
14. Tony Yaacoub, George V. Moustakides and Yajun Mei (2017), 'Optimal stopping for interval estimation in Bernoulli trials', arXiv:1711.06912, https://arxiv.org/abs/1711.06912