Climate-network methods claim binary El Niño warnings up to about one year ahead, overcoming the spring predictability barrier. Hu, Eichner, Faust and Kantz (Climate Dynamics 58:1049–1063, 2022) proposed a circular time-shift surrogate benchmark — shifting the predictive index against the verification record while preserving all of its statistical properties — and found that a naïve Oceanic Niño Index analogue forecast loses skill at moderate lead times, that the basin-spanning network index of Ludescher et al. passes the benchmark, and that the Niño3.4-only index of Meng et al. is borderline. They left open the validation and generalization of this benchmarking program: which data-design choices (spatial domain, nodes, index definition, thresholds, and the conversion rules that map a continuous index into event warnings) carry the predictive signal; how performance should be assessed at exact lead times; how the benchmark extends to multi-category and continuous forecasts and to La Niña; and how skill can be certified on records containing only a few dozen events.

Background

The El Niño–Southern Oscillation (ENSO) is the coupled ocean–atmosphere mode that dominates interannual climate variability worldwide. Its phase is tracked by the Oceanic Niño Index (ONI), the 3-month running mean of sea-surface-temperature anomalies in the Niño3.4 region of the equatorial Pacific (5°N–5°S, 170°W–120°W): an El Niño is conventionally present when the ONI stays at or above +0.5 °C for at least five consecutive months, and a La Niña when it stays at or below −0.5 °C for at least five consecutive months. Dynamical forecast skill drops sharply across boreal spring — the spring predictability barrier (SPB) — so claims of early warnings issued roughly a year before onset attract great interest. A family of statistical methods based on climate networks makes such claims: a climate network treats grid points of a reanalysis field as nodes and the cross-correlation between their time series as link strength. Ludescher, Gozolchiani, Bogachev, Bunde, Havlin and Schellnhuber (PNAS 110:11742, 2013; PNAS 111:2064, 2014) average the link strengths between 14 nodes in the equatorial Pacific and 193 nodes elsewhere into an index S(t)S(t); when S(t)S(t) crosses a threshold from below while the Niño3.4 index is still low, an El Niño onset is predicted for the following calendar year. Meng, Fan, Ashkenazy, Bunde and Havlin (New J. Phys. 20:043036, 2018) build an index FI(t)\mathrm{FI}(t) from disorder among 105 nodes inside the Niño3.4 basin alone, with a warning rule that also predicts an El Niño within 18 months after an intervening La Niña. Both schemes output binary warnings, scored by hit rate (HR, fraction of events correctly warned) and false alarm rate (FAR, fraction of non-events wrongly warned). Because only a few dozen El Niño events exist in the observational record, apparent skill could arise by chance. Surrogate-data testing addresses this by comparing the actual forecast against an ensemble of randomized versions that preserve the forecast's statistical properties; Hu, Eichner, Faust and Kantz (Clim. Dyn. 58:1049–1063, 2022) generated surrogates by circularly shifting the predictive index relative to the verification series by integer multiples of one year — sub-annual shifts would destroy the seasonality of ENSO, and shifts shorter than the one-year correlation time are excluded to ensure the forecast–verification correlation is genuinely destroyed — which leaves the index itself untouched (33 shifted series in each case). Plotting each shifted forecast in the FAR–HR plane and fitting a bivariate Gaussian, the unshifted forecast is declared skillful at 95% confidence only if it falls outside the 95% quantile ellipse on the favorable side. Under this benchmark, the ONI-analogue naïve predictor has no genuine skill at moderate lead times, Ludescher's S(t)S(t) clearly passes, and Meng's FI(t)\mathrm{FI}(t) sits at the boundary of the null ensemble. The authors also note that the complicated conversion of a continuous index into binary event warnings (threshold choices, conditions on the ENSO state, and the La Niña-cascade rule) makes it not perfectly transparent how much apparent performance is index skill and how much is the conversion rules suppressing spurious forecasts.

Problem Statement

The open problem, as posed by Hu, Eichner, Faust and Kantz (2022) for the climate-network El Niño warning family, is to turn claimed ~1-year-lead binary ENSO early warnings into statistically validated and attributable forecasts on short observational records. Its linked components, all four of which are jointly required to resolve this one objective, are: (1) systematically determine which data-design choices carry the predictive signal — the sensitivity of skill to the spatial domain and node choice, in particular the role of nodes outside the Niño3.4 basin (Ludescher's 14+193-node design versus Meng's 105-node basin-only design), to the index definition, and to the conversion rules (thresholds, ENSO-state conditions, and cascade rules after intervening La Niña events) that map a continuous index into binary warnings, quantifying how much apparent performance is genuine index skill rather than suppression of spurious forecasts; the borderline position of Meng's FI(t)\mathrm{FI}(t) at the boundary of the circular-shift surrogate null is a particularly hard instance of this attribution; (2) assess the performance of the network-based forecasts at certain exact lead times, to decrease the uncertainty and provide more information for stakeholders; (3) generalize the circular-shift surrogate benchmark from binary to multi-category and continuous (real-valued) forecasts, and establish its statistical calibration for the very short records available; and (4) extend binary-warning benchmarking to La Niña forecasts, whose prediction mechanisms differ from El Niño's because of phase asymmetry and multi-year persistence. Component (1) supplies the attribution (including the hard borderline instance of Meng's FI(t)\mathrm{FI}(t)), (2) the lead-time-resolved assessment of skill, and (3)–(4) the generalization of the validation instrument across forecast types and ENSO phases; together they answer how much of the claimed improvement in long-lead El Niño early warnings is genuine.

References

  1. Hu, Xinjia; Eichner, Jan; Faust, Eberhard; Kantz, Holger (2022): Benchmarking prediction skill in binary El Niño forecasts. Climate Dynamics 58(3–4):1049–1063. DOI 10.1007/s00382-021-05950-2, https://doi.org/10.1007/s00382-021-05950-2
  2. Ludescher, Josef; Gozolchiani, Avi; Bogachev, Mikhail I.; Bunde, Armin; Havlin, Shlomo; Schellnhuber, Hans Joachim (2013): Improved El Niño forecasting by cooperativity detection. Proceedings of the National Academy of Sciences 110(29):11742–11745. DOI 10.1073/pnas.1309353110, https://doi.org/10.1073/pnas.1309353110
  3. Ludescher, Josef; Gozolchiani, Avi; Bogachev, Mikhail I.; Bunde, Armin; Havlin, Shlomo; Schellnhuber, Hans Joachim (2014): Very early warning of next El Niño. Proceedings of the National Academy of Sciences 111(6):2064–2066. DOI 10.1073/pnas.1323058111, https://doi.org/10.1073/pnas.1323058111
  4. Meng, Jun; Fan, Jingfang; Ashkenazy, Yosef; Bunde, Armin; Havlin, Shlomo (2018): Forecasting the magnitude and onset of El Niño based on climate network. New Journal of Physics 20(4):043036. DOI 10.1088/1367-2630/aabb25, https://doi.org/10.1088/1367-2630/aabb25
  5. Bunde, Armin; Ludescher, Josef; Schellnhuber, Hans Joachim (2024): Evaluation of the real-time El Niño forecasts by the climate network approach between 2011 and present. Theoretical and Applied Climatology 155:6727–6736. DOI 10.1007/s00704-024-05035-0, https://doi.org/10.1007/s00704-024-05035-0
  6. Han, Jiangxue; Ludescher, Josef; Yuan, Naiming (2026): Can CMIP6 models simulate the network-based early warning signals for El Niño? Insights for climate modeling and network analyses. Climate Dynamics 64:72. DOI 10.1007/s00382-026-08057-8, https://doi.org/10.1007/s00382-026-08057-8
  7. Ludescher, Josef; Meng, Jun; Fan, Jingfang; Bunde, Armin; Schellnhuber, Hans Joachim (2025): Climate network and complexity approach predict neutral ENSO event for 2025. arXiv:2502.00643, https://arxiv.org/abs/2502.00643
  8. Ludescher, Josef; Meng, Jun; Fan, Jingfang; Bunde, Armin; Schellnhuber, Hans Joachim (2026): Climate network and complexity based ENSO forecast for 2026. arXiv:2602.14773, https://arxiv.org/abs/2602.14773
  9. Saha, Ruby; Ghosh, Dibakar (2025): Analysis of pre-El Niño and La Niña events using climate network approach. Chaos, Solitons & Fractals 191:115781. DOI 10.1016/j.chaos.2024.115781, https://doi.org/10.1016/j.chaos.2024.115781
  10. Ludescher, Josef; Bunde, Armin; Schellnhuber, Hans Joachim (2022): El Niño forecasting by climate networks: comparison of the forecasting performance in observational data and in historical and controls runs of CMIP5 and CMIP6. EGU General Assembly 2022, EGU22-6329. DOI 10.5194/egusphere-egu22-6329, https://doi.org/10.5194/egusphere-egu22-6329