When a flood forecast reaches flow magnitudes beyond anything seen during model calibration (an extrapolation context), the probability distributions attached to the forecast — its predictive uncertainty — often stop being reliable. Berthet et al. (2020) documented this systematically: on 154 French catchments, a statistical post-processor of rainfall–runoff forecasts showed a significant drop in reliability when forecasting high flows in extrapolation, with large variability among catchments and lead times, and none of the tested catchment or forecast characteristics correlated with the performance loss. The open problem is to identify the hydrological, model-structural, and forecast conditions under which predictive-uncertainty quantification breaks down in extrapolation, to explain the mechanism of the breakdown, and thereby to enable advance detection of unreliable uncertainty estimates during major floods.

Background

Operational flood forecasting services issue discharge forecasts from conceptual rainfall–runoff models driven by observed or forecast rainfall. For major floods, forecasts frequently enter an extrapolation context: the predicted discharge magnitude was never observed during calibration, so both the model and the statistical treatment of its errors operate outside the range where they were fitted.

Predictive uncertainty (PU) is the probability distribution of the future discharge conditional on the information available at forecast time. Following the Bayesian forecasting framework of Krzysztofowicz (1999), total uncertainty is decomposed into input uncertainty (mainly rainfall) and hydrological uncertainty (everything else); the two are often treated separately. Hydrological uncertainty is commonly quantified by statistical post-processing of the deterministic model output. The study underlying this problem used the empirical hydrological uncertainty processor (EHUP) of Bourgin et al. (2014), a non-parametric, data-based post-processor used operationally in France. Because forecast errors are heteroscedastic — their spread changes with discharge magnitude — post-processors are usually applied in a transformed flow space, using variable transformations such as the logarithm, the Box–Cox transformation (a power transform with an exponent parameter), or the log–sinh transformation (whose parameters are calibrated). The quality of a predictive distribution is judged by reliability (do the stated probabilities match observed frequencies of exceedance, assessed for example with probability integral transform, PIT, histograms — the PIT values of a perfectly reliable forecast are uniformly distributed) along with sharpness and overall scores.

Berthet et al. (2020) proposed a 'crash-testing' evaluation framework that concentrates on extrapolation episodes — events exceeding the flows encountered during calibration — and applied it to 154 French catchments with the GRP rainfall–runoff model (a parsimonious operational model) and the EHUP, testing the log, Box–Cox, and log–sinh transformations. Their findings: an appropriate transformation clearly improves predictive distributions, but a significant performance loss in extrapolation remains with any of the three transformations; more flexible transformations with calibrated parameters (log–sinh) are not more reliable in extrapolation than the uncalibrated log or Box–Cox transforms; the loss varies considerably among catchments and lead times; and none of the catchment or forecast characteristics they tested was significantly correlated with the performance loss. The mechanism of the breakdown therefore remains unexplained, and no means exists to anticipate which forecasts will be unreliable exactly when reliability matters most — during major floods.

Problem Statement

Under what hydrological and model-structural conditions does predictive-uncertainty quantification break down when forecasting high flows in an extrapolation context, and why? Berthet et al. (2020) showed that the reliability of post-processed predictive distributions drops significantly when forecasts exceed the flow magnitudes encountered during calibration, that the loss varies strongly among catchments and lead times, and that none of the catchment or forecast characteristics they tested correlates with it. The open problem is whether any catchment, regime, forecast, or model-structural characteristics govern whether extrapolated predictive uncertainty remains reliable or degrades, and which ones do, together with the mechanism of the degradation in the affected catchments, so that forecasters can detect in advance the hydrological situations in which extrapolated uncertainty estimates must be questioned. Within the same objective of building forecasting systems that are robust in extrapolation, the source poses three further directly linked companion questions that frame the problem: whether combining input (meteorological forecast) uncertainty with hydrological uncertainty changes the extrapolation reliability gap; whether improved calibration strategies such as regionalisation of the predictive distribution parameters can substantially mitigate it; and how hydrological model structure itself limits extrapolation performance.

References

  1. Berthet, L., Bourgin, F., Perrin, C., Viatgé, J., Marty, R., and Piotte, O.: A crash-testing framework for predictive uncertainty assessment when forecasting high flows in an extrapolation context. Hydrology and Earth System Sciences, 24, 2017–2041, 2020. https://doi.org/10.5194/hess-24-2017-2020
  2. Bourgin, F., Ramos, M.-H., Thirel, G., and Andréassian, V.: Investigating the interactions between data assimilation and post-processing in hydrological ensemble forecasting. Journal of Hydrology, 519, 2775–2784, 2014. https://doi.org/10.1016/j.jhydrol.2014.07.054
  3. Krzysztofowicz, R.: Bayesian theory of probabilistic forecasting via deterministic hydrologic model. Water Resources Research, 35, 2739–2750, 1999. https://doi.org/10.1029/1999WR900099
  4. McInerney, D., Thyer, M., Kavetski, D., Laugesen, R., Woldemeskel, F., Tuteja, N., and Kuczera, G.: Improving the Reliability of Sub-Seasonal Forecasts of High and Low Flows by Using a Flow-Dependent Nonparametric Model. Water Resources Research, 57, e2020WR029317, 2021. https://doi.org/10.1029/2020WR029317
  5. Klotz, D., Kratzert, F., Gauch, M., Sampson, A. K., Brandstetter, J., Klambauer, G., Hochreiter, S., and Nearing, G.: Uncertainty estimation with deep learning for rainfall–runoff modeling. Hydrology and Earth System Sciences, 26, 1673–1693, 2022. https://doi.org/10.5194/hess-26-1673-2022
  6. Astagneau, P. C., Bourgin, F., Andréassian, V., and Perrin, C.: When does a parsimonious model fail to simulate floods? Learning from the seasonality of model bias. Hydrological Sciences Journal, 66, 1288–1305, 2021. https://doi.org/10.1080/02626667.2021.1923720
  7. Astagneau, P. C., Bourgin, F., Andréassian, V., and Perrin, C.: Improving the structure of a hydrological model to forecast catchment response to intense rainfall. La Houille Blanche, 110, 2341027, 2024. https://doi.org/10.1080/27678490.2024.2341027
  8. Belleudy, A., Marty, R., Le Pape, E., Narbais-Jaureguy, D., and Zuber, F.: Vers la généralisation de la prévision hydrologique probabiliste au sein du réseau Vigicrues : estimation, évaluation et communication. La Houille Blanche, 110, 2374079, 2024. https://doi.org/10.1080/27678490.2024.2374079
  9. Skøien, J. O., Bogner, K., Salamon, P., and Wetterhall, F.: On the implementation of post-processing of runoff forecast ensembles. Journal of Hydrometeorology, 22, 2731–2749, 2021. https://doi.org/10.1175/JHM-D-21-0008.1
  10. Huang, Z., Zhao, T., Tian, Y., Chen, X., Duan, Q., and Wang, H.: Reliability of Ensemble Climatological Forecasts. Water Resources Research, 59, e2023WR034942, 2023. https://doi.org/10.1029/2023WR034942
  11. De León Pérez, D., Salazar-Galán, S., and Francés, F.: Beyond Deterministic Forecasts: A Scoping Review of Probabilistic Uncertainty Quantification in Short-to-Seasonal Hydrological Prediction. Water, 17, 2932, 2025. https://doi.org/10.3390/w17202932