Constructive completion of the multiplicity-based maximum entropy principle for general path-dependent processes

Hanel, Thurner and Gell-Mann (PNAS 2014) proved that a maximum entropy principle (MEP) exists for non-multinomial, path-dependent stochastic processes: if the logarithmic finite-difference ratios of the histogram probability can be represented in a basis of deformed-multinomial form satisfying two explicit conditions, the probability factorizes into a theta-independent multiplicity and a constraint factor, the logarithm of the multiplicity defines a generalized entropy of the (c,d) class, and extremization under generally nonlinear (escort-type) constraints yields the most likely distribution. The theorem is conditional on finding the right basis, and the paper demonstrates it only on a single path-dependent random walk. The open problem is the constructive completion of this program: a systematic, verifiable, ideally algorithmic procedure that starts from the microscopic transition rules of an arbitrary process and produces the deformation functions, multiplicity, constraint structure, and finite-N corrections (or a definitive negative answer), together with the characterization of constraint-coordinate scaling (escort versus linear constraints, aging) and validation beyond the exactly solved process classes. Later work by the same group extended the catalog of solvable classes (Polya urns, sample-space-reducing processes) and reformulated the construction as the search for adjoint representations, while explicitly acknowledging that optimality of the construction is typically undecidable in practice; the general construction problem, the finite-N theory, and efficient algorithms remain open.

Background

The maximum entropy principle (MEP) is the inference rule that selects the probability distribution maximizing an entropy functional subject to constraints representing prior information; for ordinary statistical systems (gases, independent and identically distributed (i.i.d.) processes) the entropy is Boltzmann-Gibbs-Shannon entropy, the constraints are linear expectation values, and maximization yields exponential-family distributions. The microscopic underpinning is combinatorial: for a process emitting $N$ independent symbols from an alphabet of size $W$, the number of microscopic sequences compatible with a histogram $k=(k_1,\dots,k_W)$, called the multiplicity, is the multinomial coefficient, and Boltzmann entropy is the logarithm of this multiplicity per element. This derivation breaks down for complex systems whose sequence probabilities are not invariant under permutations of the sequence: nonergodic, strongly correlated, path-dependent (transition probabilities that depend on the history of the process), and aging (statistics that keep evolving with system size or time) processes. Their statistics are non-multinomial, and whether any maximum entropy principle exists for them was unclear for decades.

Hanel and Thurner classified the entropy functionals compatible with the first three Shannon-Khinchin axioms (SK1: entropy depends only on the probabilities; SK2: entropy is maximal for the uniform distribution; SK3: adding a zero-probability state does not change entropy; the fourth axiom, additivity, is dropped), obtaining the $(c,d)$-entropy universality classes, and showed how the asymptotic growth of phase-space volume determines which generalized entropy applies to a system. Hanel, Thurner and Gell-Mann also proved a duality showing that maximizing a generalized entropy under nonlinear, so-called escort constraints (expectation values computed with the renormalized powers $p_i^q/\sum_j p_j^q$ of the distribution) is equivalent to maximizing a dual generalized entropy under ordinary linear constraints.

In the 2014 PNAS paper, Hanel, Thurner and Gell-Mann proved an existence theorem for the MEP of general processes. Consider a process parameterized by bias parameters $\theta$ that produces histograms $k$ with probability $P(k\mid\theta)$, and the logarithmic finite-difference ratios $v_i(k\mid\theta)=\log[P(k-e_i\mid\theta)/P(k\mid\theta)]$, where $e_i$ is the unit vector. If the $v_i$ can be represented in a basis of $W$ vectors built from deformed multinomials, constructed from a deformed factorial $N!_u=\prod_{n=1}^{N}u(n)$ with monotone $u$ and a monotone continuous function $T$ via $M_{u,T}(k)=N!_u/\prod_i\lfloor N\,T(k_i/N)\rfloor!_u$, and the basis satisfies (i) $\kappa_{1i}=1$ for all $i$ and (ii) the first coordinate $c_1(k\mid\theta)$ of the $v_i$ becomes a nonzero constant at the maximizing histogram $k^*$, then $P(k\mid\theta)$ factorizes as $M_{u,T}(k)\,G_{u,T}(k\mid\theta)$ into a $\theta$-independent multiplicity and a constraint factor. The generalized entropy $S[p]=(1/f(N))\log M_{u,T}(k)$ with $p=k/N$ then supports a MEP whose constraints are generally nonlinear (escort-type), and the most likely distribution $p^*$ follows by extremization. The authors demonstrated the theorem on a path-dependent random walk with deformed-factorial weights, where the MEP predicted the aging (increasingly persistent) distributions confirmed by numerical simulation.

The theorem is an existence result conditional on finding a basis of the required form. The paper itself notes that the constraint factor $G_{u,T}$ may be difficult to construct in practice (although its logarithmic derivatives suffice for maximization), that additional constraints may become necessary at intermediate sequence lengths $N$ where coordinates that must vanish asymptotically are not yet small, and that the framework extends only 'possibly' to aging systems. Follow-up work solved further specific process classes: self-reinforcing Polya urn processes (urns in which the probability of drawing a color increases with the number of times it has been drawn, a canonical path-dependent model) and sample-space-reducing (SSR) processes (driven dissipative processes whose accessible state space shrinks as they evolve), and the 2023 reformulation showed that constructing the entropy functional is equivalent to finding an adjoint representation, a reversible map of the process to one that is i.i.d. What remains missing is precisely the constructive and algorithmic content of the 2014 theorem, which is the subject of this problem.

Problem Statement

The problem is the constructive completion of the multiplicity-based maximum entropy principle of Hanel, Thurner and Gell-Mann (2014), with one common scientific objective: to turn the conditional existence theorem into a systematic, verifiable procedure that starts from the microscopic transition rules of a path-dependent (non-multinomial) stochastic process and ends with the explicit entropy functional and constraint structure, or a definitive statement that no such factorization exists. Concretely, given a process on a finite alphabet with bias parameters $\theta$ specified by its transition rules, the requested results, which jointly support this objective, are:

1. Construction and decision. A systematic, ideally algorithmic and efficient, procedure that determines the deformation functions $u$ and $T$, the multiplicity $M_{u,T}$, and the constraint factor $G_{u,T}$, and verifies the basis conditions (i) and (ii) of the 2014 theorem at finite $N$; or a criterion, checkable from the transition rules, that decides when no such factorization (equivalently, by the 2023 reformulation, no adjoint representation) exists. The procedure must go beyond hand analysis of individual examples.
2. Finite-$N$ theory. A characterization of which basis coordinates $c_j(k\mid\theta)$ are asymptotically negligible, at which sequence lengths $N$ they become negligible, and which additional constraints or finite-$N$ corrections are needed at intermediate $N$ where these coordinates are not yet small.
3. Constraint scaling. A systematic characterization of the dependence of the constraint coordinates $c_j(k\mid\theta)$ on $N$ and on the bias parameters $\theta$, determining when the resulting maximum entropy principle uses escort-type versus linear constraints, and how the inverse-temperature-like parameters evolve with $N$ (aging).
4. Validation and breadth. Efficient algorithms implementing items 1-3, and their demonstration on process classes and real-world complex systems beyond the exactly solved examples (the 2014 path-dependent random walk, Polya urn processes, sample-space-reducing processes), for instance other path-dependent dynamics, interacting particle systems, or empirical aging systems.

The four results are sought jointly, for a well-defined broad class of processes. Since items 2-4 concern the constraint coordinates, finite-$N$ corrections, and validation of an exhibited factorization, they presuppose that a factorization exists; the negative branch of item 1 (a criterion, checkable from the transition rules, that decides no such factorization, equivalently no adjoint representation, exists) therefore stands on its own, independently of items 2-4.

References

1. Rudolf Hanel, Stefan Thurner, Murray Gell-Mann (2014), "How multiplicity determines entropy and the derivation of the maximum entropy principle for complex systems", Proceedings of the National Academy of Sciences 111(19):6905-6910, doi:10.1073/pnas.1406071111, https://doi.org/10.1073/pnas.1406071111
2. Rudolf Hanel, Stefan Thurner (2011), "A comprehensive classification of complex statistical systems and an axiomatic derivation of their entropy and distribution functions", EPL 93(2):20006, doi:10.1209/0295-5075/93/20006, https://doi.org/10.1209/0295-5075/93/20006
3. Rudolf Hanel, Stefan Thurner (2011), "When do generalized entropies apply? How phase space volume determines entropy", EPL 96(5):50003, doi:10.1209/0295-5075/96/50003, https://doi.org/10.1209/0295-5075/96/50003
4. Rudolf Hanel, Stefan Thurner, Murray Gell-Mann (2012), "Generalized entropies and logarithms and their duality relations", Proceedings of the National Academy of Sciences 109(47):19151-19154, doi:10.1073/pnas.1216885109, https://doi.org/10.1073/pnas.1216885109
5. Rudolf Hanel, Bernat Corominas-Murtra, Stefan Thurner (2017), "Understanding frequency distributions of path-dependent processes with non-multinomial maximum entropy approaches", New Journal of Physics 19:033008, doi:10.1088/1367-2630/aa611d, https://doi.org/10.1088/1367-2630/aa611d
6. Rudolf Hanel, Stefan Thurner (2018), "Maximum Configuration Principle for Driven Systems with Arbitrary Driving", Entropy 20(11):838, doi:10.3390/e20110838, https://doi.org/10.3390/e20110838
7. Jan Korbel, Simon David Lindner, Rudolf Hanel, Stefan Thurner (2021), "Thermodynamics of structure-forming systems", Nature Communications 12:1127, doi:10.1038/s41467-021-21272-7, https://doi.org/10.1038/s41467-021-21272-7
8. Rudolf Hanel, Stefan Thurner (2023), "Equivalence of information production and generalised entropies in complex processes", PLOS ONE 18(9):e0290695, doi:10.1371/journal.pone.0290695, https://doi.org/10.1371/journal.pone.0290695
9. Jan Korbel (2026), "Foundations of entropy in complex systems", arXiv:2606.16312, https://arxiv.org/abs/2606.16312
10. Constantino Tsallis, Hans J. Haubold (2015), "Boltzmann-Gibbs entropy is sufficient but not necessary for the likelihood factorization required by Einstein", EPL 110(3):30005, doi:10.1209/0295-5075/110/30005, https://doi.org/10.1209/0295-5075/110/30005
11. Petr Jizba, Jan Korbel (2019), "Maximum Entropy Principle in Statistical Inference: Case for Non-Shannonian Entropies", Physical Review Letters 122:120601, doi:10.1103/PhysRevLett.122.120601, https://doi.org/10.1103/PhysRevLett.122.120601
12. Kenric P. Nelson (2024), "Open Problems within Nonextensive Statistical Mechanics", Entropy 26(2):118, doi:10.3390/e26020118, https://doi.org/10.3390/e26020118
13. Kenric P. Nelson (2026), "The unique, universal entropy for complex systems", arXiv:2605.04493, https://arxiv.org/abs/2605.04493