Physically-based rainfall-runoff models are routinely used to predict how land-use or climate change alters catchment runoff, yet the error of such predictions is a confounded mixture of model-structure error (wrong or missing process equations), parameter error (wrong parameter values in those equations), and forcing-data error, and these components cannot be isolated from the integrated responses observed at the catchment outlet. Ewen, O'Donnell, Burton and O'Connell (2006) demonstrated, using perfect-knowledge experiments on a simple model of the Slapton Wood catchment, that parameter sets conditioned on such integrated responses (as in calibration under the GLUE framework, Generalised Likelihood Uncertainty Estimation) are unreliable for predicting change effects precisely because structure error and parameter error interact. The open problem is to make predictive assessment of catchment change reliable: to separate, or at least soundly constrain, the contributions of structure, parameter and forcing error in real catchments where only imperfect forcing data and outlet responses are available. A later formal analysis showed the separation is ill-posed without independent prior information on data errors, and the community's 2019 list of twenty-three unsolved problems in hydrology still names it; the problem remains open.

Background

A rainfall-runoff model simulates the conversion of rainfall into river discharge at a catchment outlet. It ranges along a spectrum from 'lumped conceptual' models, which describe the catchment as a small number of storage compartments with simple equations and a handful of calibrated parameters, to 'distributed physically-based models' (DPBMs) such as SHETRAN, which solve coupled partial-differential equations of surface and subsurface flow on a spatial grid with many parameters intended to be measurable soil, vegetation and topographic properties. To predict the effect of a physical change in a catchment (drain installation, afforestation, urbanisation, climate shifts), the standard approach is: (i) calibrate the model on a historical period of forcing and outlet discharge data; (ii) adjust parameter values to represent the imposed change; (iii) run the changed simulation and compare with the unchanged one. Model calibration here means choosing parameter values that make simulated outlet discharge agree with observations, either by minimising an error measure or by an uncertainty-estimation framework. The most widely used such framework is GLUE (Generalised Likelihood Uncertainty Estimation; Beven and Binley, 1992), which runs the model with many random parameter sets, retains those giving acceptable fit as 'behavioural' sets according to a (subjectively chosen) likelihood threshold, and takes the spread of their simulations as the prediction uncertainty. GLUE was invented to handle equifinality, the phenomenon that many different parameter sets, and indeed many different model structures, reproduce the observed outlet hydrograph about equally well, so the calibration data cannot discriminate among them. The total ('integrated') error between simulated and observed outlet responses aggregates at least three components: model-structure error (the equations describe the catchment imperfectly), parameter error (the values used are wrong), and run-time or forcing error (the rainfall and other input data are wrong). Ewen, Parkin (1996) and others had framed how catchment models for change prediction should be validated; Ewen, O'Donnell, Burton and O'Connell (2006) then asked what calibration actually achieves for that purpose. In a 'perfect-model' (perfect-knowledge) experiment they built a deliberately simple model of the Slapton Wood research catchment (a small instrumented catchment in Devon, UK) in a synthetic world where the true catchment responses are known, so that the individual error components can be controlled and their contributions to prediction error measured. Their finding: parameter sets accepted as behavioural on integrated outlet errors are unreliable for predicting the effects of an imposed physical change, because model-structure error and parameter error interact - the calibration compensates structural deficiencies by distorting parameter values, and that distortion then propagates wrongly into the changed scenario. Related challenges were taken up in the later literature rather than resolved by that demonstration: how to propagate the uncertainty of the parameter adjustment representing the imposed change by objective rather than subjective procedures, whether the single-compartment demonstration extends to the large multi-parameter distributed physically-based models used in practice, and what field observations would make the error decomposition identifiable. This connects to a general statistical result: calibrating an imperfect computer model produces parameter estimates that absorb the model's discrepancy term (Kennedy and O'Hagan, 2001), so decomposition requires independent information. In hydrology the Bayesian total error analysis line (Kuczera, Kavetski, Franks and Thyer, 2006; Renard, Kavetski, Kuczera, Thyer and Franks, 2010) made this precise: representing storm-to-storm parameter variability as latent variables captures some structural error, but decomposing total predictive uncertainty into input and structural components is ill-posed unless sufficiently precise, independent prior knowledge of data uncertainty is supplied. Community syntheses - the Predictions in Ungauged Basins decade review (Hrachowitz et al., 2013), the GLUE retrospective (Beven and Binley, 2013), and the community-derived list of twenty-three unsolved problems in hydrology (Blöschl et al., 2019) - all keep the error-source separation question on the open agenda. The conventions a solution must respect are those of real practice: only historical forcing data (measured with error) and outlet discharge (measured with error) are typically available, and predictions of change must be delivered with uncertainty bounds meaningful enough to support management decisions.

Problem Statement

How can the error of physically-based rainfall-runoff prediction of catchment change be decomposed and constrained well enough, under the data actually available in real catchments - imperfect forcing data and integrated outlet responses only - that predictions of physical change are reliable for management decisions? Ewen et al. (2006) showed, in a perfect-knowledge setting, that behavioural parameter sets conditioned on integrated outlet errors are unreliable for predicting change effects because model-structure error and parameter error interact, and a single rainfall-runoff data set has since been shown formally to be insufficient for the decomposition (Renard et al., 2010). The open core is therefore identifiability: what additional information - additional observables, independent priors, multi-response or spatially distributed data, experimental designs - makes the contributions of model-structure error, parameter error and forcing error separately identifiable, or at least jointly constrainable, in real catchments, so that predictions of an imposed physical change carry uncertainty bounds decomposed by source and honest enough to support management decisions.

References

  1. Ewen, J., O'Donnell, G., Burton, A., O'Connell, E. (2006). Errors and uncertainty in physically-based rainfall-runoff modelling of catchment change effects. Journal of Hydrology 330(3-4), 641-650. DOI 10.1016/j.jhydrol.2006.04.024. https://doi.org/10.1016/j.jhydrol.2006.04.024
  2. Beven, K., Binley, A. (1992). The future of distributed models: Model calibration and uncertainty prediction. Hydrological Processes 6(3), 279-298. DOI 10.1002/hyp.3360060305. https://doi.org/10.1002/hyp.3360060305
  3. Ewen, J., Parkin, G. (1996). Validation of catchment models for predicting land-use and climate change impacts. 1. Method. Journal of Hydrology 175(1-4), 583-594. DOI 10.1016/S0022-1694(96)80026-6. https://doi.org/10.1016/S0022-1694(96)80026-6
  4. Kennedy, M.C., O'Hagan, A. (2001). Bayesian calibration of computer models. Journal of the Royal Statistical Society: Series B 63(3), 425-464. DOI 10.1111/1467-9868.00294. https://doi.org/10.1111/1467-9868.00294
  5. Kuczera, G., Kavetski, D., Franks, S., Thyer, M. (2006). Towards a Bayesian total error analysis of conceptual rainfall-runoff models: Characterising model error using storm-dependent parameters. Journal of Hydrology 331(1-2), 161-177. DOI 10.1016/j.jhydrol.2006.05.010. https://doi.org/10.1016/j.jhydrol.2006.05.010
  6. Breuer, L., Huisman, J.A., Frede, H.-G. (2006). Monte Carlo assessment of uncertainty in the simulated hydrological response to land use change. Environmental Modeling & Assessment 11(3), 209-218. DOI 10.1007/s10666-006-9051-9. https://doi.org/10.1007/s10666-006-9051-9
  7. Renard, B., Kavetski, D., Kuczera, G., Thyer, M., Franks, S.W. (2010). Understanding predictive uncertainty in hydrologic modeling: The challenge of identifying input and structural errors. Water Resources Research 46(5), W05521. DOI 10.1029/2009WR008328. https://doi.org/10.1029/2009WR008328
  8. Prudhomme, C., Wilby, R.L., Crooks, S., Kay, A.L., Reynard, N.S. (2010). Scenario-neutral approach to climate change impact studies: Application to flood risk. Journal of Hydrology 390(3-4), 198-209. DOI 10.1016/j.jhydrol.2010.06.043. https://doi.org/10.1016/j.jhydrol.2010.06.043
  9. Zhang, X., Hörmann, G., Gao, J., Fohrer, N. (2011). Structural uncertainty assessment in a discharge simulation model. Hydrological Sciences Journal 56(5), 854-869. DOI 10.1080/02626667.2011.587426. https://doi.org/10.1080/02626667.2011.587426
  10. McIntyre, N. (2012). Modelling the hydrological impacts of rural land use change: current state of the science and future challenges. In: Hydrology for a changing world (BHS 11th National Symposium, Exeter), 01-07. DOI 10.7558/BHS.2012.NS35. https://doi.org/10.7558/BHS.2012.NS35
  11. Hrachowitz, M., Savenije, H.H.G., Blöschl, G., McDonnell, J.J., Sivapalan, M., Pomeroy, J.W., Arheimer, B., Blume, T., Clark, M.P., Ehret, U., Fenicia, F., Freer, J.E., Gelfan, A., Gupta, H.V., Hughes, D.A., Hut, R.W., Montanari, A., Pande, S., Tetzlaff, D., Troch, P.A., Uhlenbrook, S., Wagener, T., Winsemius, H.C., Woods, R.A., Zehe, E., Cudennec, C. (2013). A decade of Predictions in Ungauged Basins (PUB) - a review. Hydrological Sciences Journal 58(6), 1198-1255. DOI 10.1080/02626667.2013.803183. https://doi.org/10.1080/02626667.2013.803183
  12. Beven, K., Binley, A. (2013). GLUE: 20 years on. Hydrological Processes 28(24), 5897-5918. DOI 10.1002/hyp.10082. https://doi.org/10.1002/hyp.10082
  13. Gong, W., Gupta, H.V., Yang, D., Sricharan, K., Hero, A.O. (2013). Estimating epistemic and aleatory uncertainties during hydrologic modeling: An information theoretic approach. Water Resources Research 49(4), 2253-2273. DOI 10.1002/wrcr.20161. https://doi.org/10.1002/wrcr.20161
  14. Mockler, E.M., Chun, K.P., Sapriza-Azuri, G., Bruen, M., Wheater, H.S. (2016). Assessing the relative importance of parameter and forcing uncertainty and their interactions in conceptual hydrological model simulations. Advances in Water Resources 97, 299-313. DOI 10.1016/j.advwatres.2016.10.008. https://doi.org/10.1016/j.advwatres.2016.10.008
  15. Guo, D., Westra, S., Maier, H.R. (2017). Impact of evapotranspiration process representation on runoff projections from conceptual rainfall-runoff models. Water Resources Research 53(1), 435-454. DOI 10.1002/2016WR019627. https://doi.org/10.1002/2016WR019627
  16. Op de Hipt, F., Diekkrüger, B., Steup, G., Yira, Y., Hoffmann, H., Rode, M. (2017). Applying SHETRAN in a Tropical West African Catchment (Dano, Burkina Faso) - Calibration, Validation, Uncertainty Assessment. Water 9(2), 101. DOI 10.3390/w9020101. https://doi.org/10.3390/w9020101
  17. Blöschl, G., Bierkens, M.F.P., Chambel, A., Cudennec, C., Destouni, G., Fiori, A., Kirchner, J.W., McDonnell, J.J., Savenije, H.H.G., Sivapalan, M., Stumpp, C., Toth, E., Volpi, E., Carr, G., Lupton, C., Salinas, J., Széles, B., Viglione, A., Aksoy, H., Allen, S.T., Amin, A., Andréassian, V., Arheimer, B., Aryal, S.K., Baker, V., Bardsley, E., Barendrecht, M.H., Bartosova, A., Batelaan, O., Berghuijs, W.R., Beven, K., Blume, T., Bogaard, T., Borges de Amorim, P., Böttcher, M.E., Boulet, G., Breinl, K., Brilly, M., Brocca, L., Buytaert, W., Castellarin, A., Castelletti, A., Chen, X., Chen, Y., Chen, Y., Chifflard, P., Claps, P., Clark, M.P., Collins, A.L., Croke, B., Dathe, A., David, P.C., de Barros, F.P.J., de Rooij, G., Di Baldassarre, G., Driscoll, J.M., Duethmann, D., Dwivedi, R., Eris, E., Farmer, W.H., Feiccabrino, J., Ferguson, G., Ferrari, E., Ferraris, S., Fersch, B., Finger, D., Foglia, L., Fowler, K., Gartsman, B., Gascoin, S., Gaume, E., Gelfan, A., Geris, J., Gharari, S., Gleeson, T., Glendell, M., Gonzalez Bevacqua, A., González-Dugo, M.P., Grimaldi, S., Gupta, A.B., Guse, B., Han, D., Hannah, D., Harpold, A., Haun, S., Heal, K., Helfricht, K., Herrnegger, M., Hipsey, M., Hlaváčiková, H., Hohmann, C., Holko, L., Hopkinson, C., Hrachowitz, M., Illangasekare, T.H., Inam, A., Innocente, C., Istanbulluoglu, E., Jarihani, B., Kalantari, Z., Kalvans, A., Khanal, S., Khatami, S., Kiesel, J., Kirkby, M., Knoben, W., Kochanek, K., Kohnová, S., Kolechkina, A., Krause, S., Kreamer, D., Kreibich, H., Kunstmann, H., Lange, H., Liberato, M.L.R., Lindquist, E., Link, T., Liu, J., Loucks, D.P., Luce, C., Mahé, G., Makarieva, O., Malard, J., Mashtayeva, S., Maskey, S., Mas-Pla, J., Mavrova-Guirguinova, M., Mazzoleni, M., Mernild, S., Misstear, B.D., Montanari, A., Müller-Thomy, H., Nabizadeh, A., Nardi, F., Neale, C., Nesterova, N., Nurtaev, B., Odongo, V.O., Panda, S., Pande, S., Pang, Z., Papacharalampous, G., Perrin, C., Pfister, L., Pimentel, R., Polo, M.J., Post, D., Prieto Sierra, C., Ramos, M., Renner, M., Reynolds, J.E., Ridolfi, E., Rigon, R., Riva, M., Robertson, D.E., Rosso, R., Roy, T., Sá, J.H.M., Salvadori, G., Sandells, M., Schaefli, B., Schumann, A., Scolobig, A., Seibert, J., Servat, E., Shafiei, M., Sharma, A., Sidibe, M., Sidle, R.C., Skaugen, T., Smith, H., Spiessl, S.M., Stein, L., Steinsland, I., Strasser, U., Su, B., Szolgay, J., Tarboton, D., Tauro, F., Thirel, G., Tian, F., Tong, R., Tussupova, K., Tyralis, H., Uijlenhoet, R., van Beek, R., van der Ent, R.J., van der Ploeg, M., Van Loon, A.F., van Meerveld, I., van Nooijen, R., van Oel, P.R., Vidal, J., von Freyberg, J., Vorogushyn, S., Wachniew, P., Wade, A.J., Ward, P., Westerberg, I.K., White, C., Wood, E.F., Woods, R., Xu, Z., Yilmaz, K.K., Zhang, Y. (2019). Twenty-three unsolved problems in hydrology (UPH) - a community perspective. Hydrological Sciences Journal 64(10), 1141-1158. DOI 10.1080/02626667.2019.1620507. https://doi.org/10.1080/02626667.2019.1620507
  18. Addor, N., Melsen, L.A. (2019). Legacy, rather than adequacy, drives the selection of hydrological models. Water Resources Research 55(1), 378-390. DOI 10.1029/2018WR022958. https://doi.org/10.1029/2018WR022958
  19. Gupta, A., Mallya, G., Govindaraju, R. (2021). Separation of Structural and Measurement Uncertainties in Watershed Hydrological Models. EGU General Assembly 2021, abstract EGU21-993. DOI 10.5194/egusphere-egu21-993. https://doi.org/10.5194/egusphere-egu21-993
  20. Beven, K., Asadullah, A., Bates, P., Blyth, E., Chappell, N., Child, S., Cloke, H., Dadson, S., Everard, N., Fowler, H.J., Freer, J., Hannah, D.M., Heppell, K., Holden, J., Lamb, R., Lewis, H., Morgan, G., Parry, L., Wagener, T. (2020). Developing observational methods to drive future hydrological science: Can we make a start as a community? Hydrological Processes 34(3), 868-873. DOI 10.1002/hyp.13622. https://doi.org/10.1002/hyp.13622