Rigorous error bounds for entropy-production estimation from partially observed trajectories with unobserved transitions

Estimating the entropy production rate of a Markov jump process from a finite trajectory is corrupted by lacking data: reverse transitions too rare to be observed make the direct flux-ratio estimate diverge. Baiesi, Nishiyama, and Falasco (Communications Physics 7, 264 (2024)) proposed lower-bound estimators for this regime, combining an improved thermodynamic uncertainty relation involving the inverse hyperbolic tangent, the hyperaccurate-current precision, and the frenesy (dynamical activity); when all observed transitions appear unidirectional they regularize by assuming the total unobserved reverse rate is of order $1/t$. These estimators outperform direct and Bayesian approaches far from equilibrium, but rest on uncontrolled assumptions: the $1/t$ substitution has no error bound, edges unobserved in both directions are dismissed without proof, the short-time limit exploited in the derivation lacks finite-time and finite-sample corrections, and the analysis assumes the network remains ergodic after removing unobserved edges. The problem of certified error control for entropy-production estimation with unobserved transitions remains open.

Background

In stochastic thermodynamics, a mesoscopic system hopping between discrete states is modeled as a stationary Markov jump process with transition rates $w_{ij}$ from state $i$ to state $j$ and steady-state probabilities $\rho_i$. Under local detailed balance ($w_{ij}/w_{ji}=e^{s_{ij}}$, where $s_{ij}=-s_{ji}$ is the entropy deposited in the environment when transition $(i,j)$ fires, in units of $k_B$), the mean entropy production rate is $\sigma = k_B\sum_{i<j}(\phi_{ij}-\phi_{ji})\ln(\phi_{ij}/\phi_{ji})$ with fluxes $\phi_{ij}=\rho_i w_{ij}$. This quantity measures time irreversibility and dissipation and is a central target of inference in experiments on molecular machines, active biological matter, and mesoscopic devices.

Direct estimation needs all fluxes: one counts transitions $n_{ij}$ over an observation time $t$ and forms empirical fluxes $\dot n_{ij}=n_{ij}/t$. The difficulty addressed by the source paper is the lacking-data regime: when $t$ is shorter than the slowest transition time, some reverse transitions are never observed ($n_{ij}=0$ while $n_{ji}\neq 0$), the process appears absolutely irreversible, and the logarithm in the direct estimate diverges. Earlier remedies replace each unobserved flux by a fictitious rate of order $t^{-1}$ — heuristically, no transition was seen in time $t$ because its flux was barely below $1/t$ — either uniformly or in the Bayesian refinement $\dot n_{ij}\simeq p_j/(p_i t)$ of Zeraati, Jafarpour, and Hinrichsen, and insert the regularized fluxes into the direct formula.

Baiesi, Nishiyama, and Falasco instead built a lower-bound estimator that avoids the direct formula. The thermodynamic uncertainty relation (TUR) bounds $\sigma$ from below using only the mean and variance of an integrated current; its optimized version uses the hyperaccurate current, a current whose short-time precision saturates the TUR and which is characterized by $p^{hyp}=\sum_{i<j}(\phi_{ij}-\phi_{ji})^2/(\phi_{ij}+\phi_{ji})$, and is tightest in the short-time ($\tau\to 0$) limit of the sampling interval. Combining this with the unified thermodynamic–kinetic uncertainty relation of Vo, Van Vu, and Hasegawa, which improves the TUR through an inverse-hyperbolic-tangent factor and involves the frenesy (dynamical activity) $\kappa=\sum_{i<j}(\phi_{ij}+\phi_{ji})$ — the time-symmetric total rate of agitation of the system — yields their bound (written, as in the source, in units $k_B=1$)

$$\sigma \;\ge\; 2\sqrt{p^{hyp}\kappa}\;\tanh^{-1}\!\sqrt{p^{hyp}/\kappa}.$$

When every observed edge appears unidirectional, $p^{hyp}_{emp}=\kappa_{emp}$ and the bound diverges; they regularize by assuming the observation time was barely longer than the typical time for a reverse transition on any of the $|\mathcal{I}|$ unobserved edges, i.e. that the total unobserved reverse rate is of order $1/t$. In the published version this gives $p^{hyp}/\kappa\simeq 1-4/(\kappa t)$ and hence a frenesy-only bound $\sigma/k_B \ge 2\kappa\tanh^{-1}\sqrt{1-4/(\kappa t)} \simeq \kappa\ln(\kappa t)$ for $\kappa t\gg 1$. The resulting estimator $\sigma^{hyp}_{tanh}$ outperforms direct and Bayesian estimates in strongly irreversible regimes and extends to deterministic limits such as irreversible chemical reactions. The derivation, however, rests on uncontrolled assumptions: the $\sim t^{-1}$ substitution is heuristic with no error bound; the short-time $\tau\to 0$ precision is exploited without finite-time or finite-sample corrections; the analysis assumes the network stays connected (ergodic) after deleting the unobserved-transition edges; and edges unobserved in both directions ($n_{ij}=n_{ji}=0$) are dismissed in one sentence — one neglects the edge, at the possible price of underestimating $\sigma$ — without proof.

Problem Statement

Provide rigorous error control for entropy-production inference from partially observed trajectories of Markov jump processes with unobserved transitions — turning the Baiesi–Nishiyama–Falasco lower-bound scheme (or a principled successor) into estimation with certified guarantees. The single objective is a provable validity and tightness statement, using only information available from the observed record, for dissipation inference in the lacking-data regime; its four linked components are:

1. Controlled regularization. Replace the heuristic assumption that the total unobserved reverse rate is of order $t^{-1}$ with a controlled procedure: for an arbitrary number of unobserved reverse transitions of possibly different scales, quantify with provable error bounds (or certified conservativeness) the error incurred by substituting fictitious unobserved rates, or prove that the substitution can fail and characterize when.

2. Doubly-unobserved edges. Give a rigorous treatment of edges with $n_{ij}=n_{ji}=0$: prove or disprove that evaluating the bound on the residual graph (neglecting such edges) remains a valid lower bound on the total $\sigma$, and quantify the resulting loss of tightness.

3. Finite-time and finite-sample theory. Derive confidence intervals for the estimator $\sigma^{hyp}_{tanh}$ that account both for statistical fluctuations of the empirical counts at finite observation time $t$ and for the deviation of the hyperaccurate-current precision from its $\tau\to 0$ limit at finite sampling interval $\tau$.

4. Connectivity assumption. Extend the analysis to observation graphs that become disconnected when the unobserved-transition edges are removed, or establish what cannot be inferred about $\sigma$ in that case.

A complete answer establishes, under assumptions that use only observed-record information — barring access to the true hidden rates or unobservable fluxes, while explicitly declared priors or distribution-free concentration statements over the hidden rates remain admissible — a certified error bound or confidence statement for entropy-production estimation under unobserved transitions covering these components as one coherent guarantee. Where a component is resolved by a rigorous negative outcome (a proof that the substitution can fail, a disproof of subgraph validity, or an impossibility characterization for disconnected observation graphs), that outcome enters the guarantee as a conditional or piecewise validity-and-tightness statement. Partial progress on individual components (for example, finite-sample confidence intervals alone) constitutes partial progress toward the objective, not a full resolution.

References

1. M. Baiesi, T. Nishiyama, and G. Falasco, 'Effective estimation of entropy production with lacking data', Communications Physics 7, 264 (2024), DOI 10.1038/s42005-024-01742-2, arXiv:2305.04657, https://doi.org/10.1038/s42005-024-01742-2
2. U. Seifert, 'Universal bounds on entropy production from fluctuating coarse-grained trajectories', Nature Reviews Physics 8, 493–507 (2026), DOI 10.1038/s42254-026-00954-5, arXiv:2512.07772, https://doi.org/10.1038/s42254-026-00954-5
3. J. H. Fritz, B. Ertel, and U. Seifert, 'Entropy estimation for partially accessible Markov networks based on imperfect observations: Role of finite resolution and finite statistics', Physical Review E 111, 044106 (2025), DOI 10.1103/PhysRevE.111.044106, arXiv:2412.04102, https://doi.org/10.1103/PhysRevE.111.044106
4. R. Bebon and A. Godec, 'Controlling Uncertainty of Empirical First-Passage Times in the Small-Sample Regime', Physical Review Letters 131, 237101 (2023), DOI 10.1103/PhysRevLett.131.237101, arXiv:2301.08732, https://doi.org/10.1103/PhysRevLett.131.237101
5. Van Tuan Vo, Tan Van Vu, and Yoshihiko Hasegawa, 'Unified thermodynamic–kinetic uncertainty relation', Journal of Physics A: Mathematical and Theoretical 55, 405004 (2022), DOI 10.1088/1751-8121/ac9099, arXiv:2203.11501, https://doi.org/10.1088/1751-8121/ac9099
6. S. Zeraati, F. H. Jafarpour, and H. Hinrichsen, 'Entropy production of nonequilibrium steady states with irreversible transitions', Journal of Statistical Mechanics: Theory and Experiment 2012, L12001 (2012), DOI 10.1088/1742-5468/2012/12/l12001, https://doi.org/10.1088/1742-5468/2012/12/l12001
7. P. E. Harunari, 'Uncovering nonequilibrium from unresolved events', Physical Review E 110, 024122 (2024), DOI 10.1103/PhysRevE.110.024122, arXiv:2402.00837, https://doi.org/10.1103/PhysRevE.110.024122
8. Q. Yu and P. E. Harunari, 'Dissipation at limited resolutions: power law and detection of hidden dissipative scales', Journal of Statistical Mechanics: Theory and Experiment 2024, 103201 (2024), DOI 10.1088/1742-5468/ad8152, arXiv:2407.13707, https://doi.org/10.1088/1742-5468/ad8152
9. A. M. Maier, J. Degünther, J. van der Meer, and U. Seifert, 'Inferring kinetics and entropy production from observable transitions in partially accessible, periodically driven Markov networks', Journal of Statistical Physics 191, 104 (2024), DOI 10.1007/s10955-024-03315-7, arXiv:2406.12657, https://doi.org/10.1007/s10955-024-03315-7