Analytic determination of the oscillatory-to-chaotic transition in fully connected asymmetric neural networks: closing probabilities and the critical symmetry

Deterministic parallel dynamics on a fully connected network of $N$ binary neurons with quenched couplings of symmetry $\eta=\langle J_{ij}J_{ji}\rangle/\langle J_{ij}^2\rangle$ interpolates between purely oscillatory behavior at high symmetry, where typical attractors are limit cycles of length 2, and chaotic behavior at low symmetry, where attractor lengths grow exponentially with $N$ and attraction-basin weights are distributed as in a random map with reversal symmetry. Bastolla and Parisi (1998) located this sharp transition numerically at $\eta_c=0.33\pm0.01$ and traced its mechanism to the closing probabilities, i.e. the conditional probabilities that the overlap between the configurations at times $t$ and $t+l$ equals $\pm1$ given that no configuration has yet been repeated, whose large-$N$ exponents $\alpha_2(\eta)$ (cycle length $l=2$) and $\alpha_\infty(\eta)$ (large $l$) should balance as $\alpha_\infty=2\alpha_2$ at the transition. They could not compute these exponents because that requires the overlap distribution of the infinite system at general symmetry, which is unknown, and they could not even confirm that $\alpha_\infty(\eta)$ exists, since the lag at which the measured closing probability appears stationary grows with system size. The problem is to determine this transition from first principles: compute the closing probabilities (equivalently, the conditional overlap distribution) at general symmetry, decide whether the asymptotic exponent $\alpha_\infty(\eta)$ exists, and derive the critical symmetry $\eta_c$ — or establish that the transition picture must be modified. Exact results known to date cover only the fully asymmetric limit, and later cycle-counting formalisms cannot access basin weights, so the basin-weighted transition still has no analytic determination.

Background

Associative-memory networks — networks of binary neurons that retrieve stored patterns from partial or noisy cues — with random synaptic couplings generalize the classical symmetric framework (Hopfield-type networks with symmetric synapses) by allowing asymmetric couplings, which is both biologically realistic and dynamically consequential. In the simplest fully connected version, $N$ binary neurons (spins) $\sigma_i\in\{-1,+1\}$ are updated simultaneously by the deterministic parallel rule

$$\sigma_i(t+1)=\mathrm{sgn}\left(\sum_{j}J_{ij}\sigma_j(t)\right),$$

where the couplings $J_{ij}$ are quenched random variables (drawn once and frozen during the dynamics), chosen so that the symmetry parameter $\eta=\langle J_{ij}J_{ji}\rangle/\langle J_{ij}^2\rangle$ takes a prescribed value: $\eta=1$ gives symmetric couplings, $\eta=0$ fully asymmetric (statistically independent) couplings, and $\eta=-1$ antisymmetric couplings. Because the state space is finite ($2^N$ configurations), every trajectory eventually repeats a configuration and settles onto a fixed point or onto a limit cycle, a periodic sequence of states. The reference model for the statistics of such attractors is the random map, a uniformly random assignment of a successor to each of the $2^N$ states, for which the distribution of attractor lengths and of attraction-basin weights (the fractions of initial conditions that flow to each attractor) is known exactly.

Work since the mid-1980s established that asymmetry qualitatively changes the attractors of this model. Numerical studies found that below some symmetry the typical cycle length grows exponentially with $N$ (chaotic attractors), while at high symmetry typical attractors are cycles of length 2; the exponential growth rate of the number of fixed points was computed analytically in the Tanaka-Edwards formalism (a mean-field statistical-mechanics count of the states of a random network); and the fully asymmetric case $\eta=0$ was solved by Bastolla and Parisi, who showed that the attraction-basin weights then follow the statistics of a random map with reversal symmetry (the global spin-flip symmetry of the dynamics), with moments

$$\overline{Y_{n+1}}=\frac{1}{2}\frac{(n!)^2}{(2n+1)!}\left(4^n+2^n\right),$$

where $Y_n=\sum_\alpha W_\alpha^n$ sums the $n$-th power of the basin weights $W_\alpha$. For the infinite system, the remanent magnetization — the long-time overlap of the state with its initial condition — vanishes below $\eta\simeq0.825$ and is finite above it; this loss-of-memory transition is distinct from the attractor transition studied here.

Bastolla and Parisi (1998) analyzed general symmetry. Their central objects are the closing probabilities: the conditional probability $\pi_N^\pm(t,t+l)$ that the overlap $q(t,t+l)=\frac{1}{N}\sum_i\sigma_i(t)\sigma_i(t+l)$ between the configurations at times $t$ and $t+l$ equals $\pm1$ (closure onto a cycle of length $l$ or $2l$), conditioned on no configuration having been repeated before time $t+l$ (the opening condition). They argued that the overlap distribution has the large-deviation form $\Pr\{q(t,t+l)=q\}\approx A\,N^{-1/2}\exp(-N\alpha(q;\eta))$, so that each closing probability decays exponentially in $N$, and that the transition between the oscillatory phase (typical attractors are length-2 cycles) and the chaotic phase (exponentially long attractors with random-map basin statistics) occurs where the exponent $\alpha_2(\eta)$ of the asymptotic closing probability for $l=2$ satisfies $\alpha_\infty(\eta)=2\alpha_2(\eta)$, with $\alpha_\infty$ the corresponding exponent at large $l$. Using as order parameters the probability $P(2)$ that a random initial condition reaches a length-2 cycle and the average basin weight $\overline{Y_2}$ (the probability that two random initial conditions reach the same attractor), they located the transition numerically at $\eta_c=0.33\pm0.01$, well below the average-cycle-length threshold $\eta_L\approx0.5$ and the remanent-magnetization threshold; they also found that $P(2)$ tends to overestimate and $\overline{Y_2}$ to underestimate the threshold, with slow convergence in $N$.

The obstacle they identified is that evaluating the exponents requires the distribution of the overlap in the infinite system at general symmetry under the opening condition, which is not known. In the fully asymmetric case the annealed approximation — treating successive overlaps as a Markov chain, later supported by a dynamical mean-field theory (a self-consistent treatment of the infinite-system dynamics) developed by Toyoizumi and Huang — works; at partial symmetry the effective ferromagnetic coupling between time slices makes the overlap dynamics non-Markovian (history-dependent), and no exact solution is available at partial symmetry. This gap leaves the critical symmetry $\eta_c$ without a first-principles value, and the possible non-existence of $\alpha_\infty(\eta)$ (the measured closing probability appears stationary only at lags that grow with $N$) leaves even the formulation of the transition condition in doubt.

Problem Statement

Determine, from first principles, the oscillatory-to-chaotic transition of the fully connected binary neural network with deterministic parallel dynamics and quenched couplings of symmetry $\eta$. The objective is a quantitative analytic theory of the transition, comprising three linked deliverables:

1. Compute the closing probabilities $\pi_N^\pm(t,t+l;\eta)$ — the conditional probabilities, given that no configuration has been repeated along the trajectory before time $t+l$ (the opening condition), that the overlap $q(t,t+l)=\frac{1}{N}\sum_i\sigma_i(t)\sigma_i(t+l)$ between the configurations at times $t$ and $t+l$ equals $+1$ or $-1$, corresponding to closure onto a cycle of length $l$ or $2l$ — or equivalently the distribution of this overlap under the opening condition, in the $N\to\infty$ limit at general symmetry $\eta$, and extract the exponents $\alpha_2(\eta)$ (asymptotic in $t$, cycle length $l=2$) and $\alpha_\infty(\eta)$ (large $l$) governing their exponential decay in $N$.

2. Establish whether the asymptotic exponent $\alpha_\infty(\eta)$ exists at all — that is, whether the asymptotic closing probability $\pi_N^*(l;\eta)$ becomes stationary in the cycle length $l$ at fixed large $N$ — or prove that it does not and reformulate the transition condition accordingly.

3. Derive the critical symmetry $\eta_c$ at which the basin-weighted transition takes place (where $\alpha_\infty=2\alpha_2$, the probability $P(2)$ that a random initial condition reaches a length-2 cycle drops from 1 to 0, and the average attraction-basin weight $\overline{Y_2}$ — the probability that two random initial conditions flow to the same attractor — rises from 0 to the random-map value 1/2), to be compared with the numerical estimate $\eta_c=0.33\pm0.01$.

Alternatively, establish that this transition picture must be modified (for example, that the basin-weighted transition is not sharp in the thermodynamic limit, or that the balance condition $\alpha_\infty=2\alpha_2$ is ill-posed because $\alpha_\infty$ does not exist).

References

1. Ugo Bastolla and Giorgio Parisi, "Relaxation, closing probabilities and transition from oscillatory to chaotic attractors in asymmetric neural networks", J. Phys. A: Math. Gen. 31, 4583-4602 (1998). DOI: 10.1088/0305-4470/31/20/003, https://doi.org/10.1088/0305-4470/31/20/003 (source paper; arXiv:cond-mat/9803224)
2. Sungmin Hwang, Viola Folli, Enrico Lanza, Giorgio Parisi, Giancarlo Ruocco, Francesco Zamponi, "On the number of limit cycles in asymmetric neural networks", J. Stat. Mech. (2019) 053402. DOI: 10.1088/1742-5468/ab11e3, https://doi.org/10.1088/1742-5468/ab11e3 (arXiv:1810.09325)
3. Taro Toyoizumi, Haiping Huang, "Structure of attractors in randomly connected networks", Phys. Rev. E 91, 032802 (2015). DOI: 10.1103/PhysRevE.91.032802, https://doi.org/10.1103/PhysRevE.91.032802 (arXiv:1502.01091)
4. Sungmin Hwang, Enrico Lanza, Giorgio Parisi, Jacopo Rocchi, Giancarlo Ruocco, Francesco Zamponi, "On the Number of Limit Cycles in Diluted Neural Networks", J. Stat. Phys. 181, 2304-2321 (2020). DOI: 10.1007/s10955-020-02664-3, https://doi.org/10.1007/s10955-020-02664-3 (arXiv:2001.00262)
5. Viola Folli, Giorgio Gosti, Marco Leonetti, Giancarlo Ruocco, "Effect of dilution in asymmetric recurrent neural networks", Neural Networks 104, 50-59 (2018). DOI: 10.1016/j.neunet.2018.04.003, https://doi.org/10.1016/j.neunet.2018.04.003 (arXiv:1805.03886)
6. Miguel Aguilera, Masanao Igarashi, Hideaki Shimazaki, "Nonequilibrium thermodynamics of the asymmetric Sherrington-Kirkpatrick model", Nature Communications 14, 3685 (2023). DOI: 10.1038/s41467-023-39107-y, https://doi.org/10.1038/s41467-023-39107-y (arXiv:2205.09886)
7. Vittorio Erba, Freya Behrens, Florent Krzakala, Lenka Zdeborová, "Quenches in the Sherrington-Kirkpatrick model", J. Stat. Mech. (2024) 083302. DOI: 10.1088/1742-5468/ad685a, https://doi.org/10.1088/1742-5468/ad685a (arXiv:2405.04267)
8. Jérôme Garnier-Brun, Michael Benzaquen, Jean-Philippe Bouchaud, "Unlearnable Games and 'Satisficing' Decisions: A Simple Model for a Complex World", Phys. Rev. X 14, 021039 (2024). DOI: 10.1103/PhysRevX.14.021039, https://doi.org/10.1103/PhysRevX.14.021039 (arXiv:2312.12252)
9. H. Gutfreund, J. D. Reger, A. P. Young, "The nature of attractors in an asymmetric spin glass with deterministic dynamics", J. Phys. A: Math. Gen. 21, 2775-2797 (1988). DOI: 10.1088/0305-4470/21/12/020, https://doi.org/10.1088/0305-4470/21/12/020
10. K. Nützel, "The length of attractors in asymmetric random neural networks with deterministic dynamics", J. Phys. A: Math. Gen. 24, L151-L157 (1991). DOI: 10.1088/0305-4470/24/3/010, https://doi.org/10.1088/0305-4470/24/3/010
11. Ugo Bastolla and Giorgio Parisi, "Attractors in fully asymmetric neural networks", J. Phys. A: Math. Gen. 30, 5613-5631 (1997). DOI: 10.1088/0305-4470/30/16/007, https://doi.org/10.1088/0305-4470/30/16/007
12. A. Crisanti, M. Falcioni, A. Vulpiani, "Transition from regular to complex behaviour in a discrete deterministic asymmetric neural network model", J. Phys. A: Math. Gen. 26, 3441-3453 (1993). DOI: 10.1088/0305-4470/26/14/011, https://doi.org/10.1088/0305-4470/26/14/011
13. F. Tanaka, S. F. Edwards, "Analytic theory of the ground state properties of a spin glass. I. Ising spin glass", J. Phys. F: Met. Phys. 10, 2769-2778 (1980). DOI: 10.1088/0305-4608/10/12/017, https://doi.org/10.1088/0305-4608/10/12/017
14. Giorgio Parisi, "Asymmetric neural networks and the process of learning", J. Phys. A: Math. Gen. 19, L675-L680 (1986). DOI: 10.1088/0305-4470/19/11/005, https://doi.org/10.1088/0305-4470/19/11/005
15. B. Derrida, H. Flyvbjerg, "The random map model: a disordered model with deterministic dynamics", J. Physique 48, 971-978 (1987). DOI: 10.1051/jphys:01987004806097100, https://doi.org/10.1051/jphys:01987004806097100
16. T. Pfenning, H. Rieger, M. Schreckenberg, "Numerical investigation of the asymmetric SK-model with deterministic dynamics", J. Phys. I France 1, 323-329 (1991). DOI: 10.1051/jp1:1991134, https://doi.org/10.1051/jp1:1991134
17. H. Eissfeller, M. Opper, "Mean-field Monte Carlo approach to the Sherrington-Kirkpatrick model with asymmetric couplings", Phys. Rev. E 50, 709-720 (1994). DOI: 10.1103/PhysRevE.50.709, https://doi.org/10.1103/PhysRevE.50.709
18. Carles Martorell, Rubén Calvo, Alessia Annibale, Miguel A. Muñoz, "Dynamically selected steady states and criticality in non-reciprocal networks", Chaos, Solitons & Fractals 182, 114809 (2024). DOI: 10.1016/j.chaos.2024.114809, https://doi.org/10.1016/j.chaos.2024.114809 (arXiv:2312.12039)
19. S. Risau-Gusman, A. Martinez, O. Kinouchi, "Escaping from cycles through a glass transition", Phys. Rev. E 68, 016104 (2003). DOI: 10.1103/PhysRevE.68.016104, https://doi.org/10.1103/PhysRevE.68.016104
20. Aakash Kumar, Anatoly Khina, Frederik Mallmann-Trenn, Emanuele Natale, "Beyond Fixed Points: Superpolynomial Capacity of Asymmetric Hopfield Networks", arXiv:2605.24611 (2026). https://arxiv.org/abs/2605.24611