This problem concerns the open questions left by Christopher Hood's 2006 study 'Gaming in Targetworld' of the post-1998 UK regime for managing public services by quantified performance targets. The study found the three classic gaming types (ratchet effects, threshold effects, output distortion) present and documented a weak central verification posture with no coherent anti-gaming strategy, but it left open how much of the reported improvement was attributable to the targets themselves as opposed to other changes that took place at the same time — most notably increased spending on public services; how far the reported performance improvements reflected real underlying improvement, that is, how prevalent and how distributed gaming behaviors are across delivery organizations and how to measure them independently of the provider-reported data that gaming contaminates; and how target-and-performance-indicator systems can be fine-tuned or game-proofed — which of the antidotes the study enumerates (tightened rules and data definitions, modified targets or key performance indicators, audit investigations and oversight bodies, threats of discipline, mystery shoppers, third-party user-satisfaction surveys, information-system architecture limiting creative responses to data entry, converting achieved targets into permanent threshold standards) check ratchet, threshold and output-distortion gaming when applied singly or in combination. The study also recorded, without claiming the evidence to decide it, the ambiguity whether weak central verification was a deliberate tolerance of good-news data or the priorities of a government in a hurry to show results. Later literature has partially answered the causal question for NHS waiting-time targets and measured specific gaming channels such as Quality and Outcomes Framework exception reporting, but independent system-wide measurement of gaming prevalence and validated anti-gaming design configurations remain open — the game-proofing question having been revisited, in a related but differently framed form, by Hood and Piotrowska in 2024.

Background

From 1998 the UK government managed public services through a centrally negotiated system of quantified performance targets: high-level Public Service Agreement (PSA) targets agreed between the Treasury and spending departments, cascaded to delivery organizations and monitored by the Prime Minister's Delivery Unit, reinforced by public rankings such as hospital 'star ratings' and sanctions for failure. Christopher Hood's article 'Gaming in Targetworld' (Public Administration Review, 2006) asked whether this centralized regime reproduced the gaming pathologies historically associated with Soviet-style planning — a concern related to Goodhart's law, the observation that once a measure becomes a target it ceases to be a good measure. Here 'gaming' means behavior that improves the measured indicator without improving (or while degrading) the underlying service. The classic taxonomy distinguishes: ratchet effects, in which providers keep future targets easy, for example by negotiating undemanding targets or withholding performance information so targets are not tightened; threshold effects, in which effort concentrates just at the target point, for example 'storming' hospital admissions just inside the four-hour emergency-department limit; and output distortion, in which effort is reallocated or cases are creatively classified so the indicator improves while real service deteriorates, for example reclassifying patients as 'exceptions' under the primary-care Quality and Outcomes Framework (QOF) pay-for-performance contract, or heaping reported waiting times just below target thresholds. Drawing on interviews with 89 senior central-government officials and documentary audit evidence, Hood concluded that all three gaming types could be identified under the regime and — the striking finding — that central managers invested little in checking reported performance data, took reported gains largely at face value, and had no coherent anti-gaming strategy. The companion study by Bevan and Hood (Public Administration, 2006) assembled the English health-sector evidence in detail, including discrepancies between provider-reported waits and independent patient surveys. The study thus left a set of linked unresolved questions about whether target-based governance can produce genuine rather than partly illusory performance improvement.

Problem Statement

The questions that Hood's study of the UK target regime left open, sharing one scientific objective — establishing whether target-based performance management of public services produces genuine rather than partly illusory improvement, and under what institutional conditions it can do so. The open questions below are complementary components of that single objective rather than alternative routes to it: answering any one of them leaves the others open. The study posed 'three crucial questions' about the reported performance improvements (p. 516), answered each only partially at what it called the 'balance-of-probability level', and left the design question standing in its Conclusion (p. 520): (1) Causal attribution: how much of the measured improvement in targeted outcomes is attributable to the target regime itself, and how much to other changes that took place at the same time — most notably increased spending on public services? The study concluded that the reported gains could by no means be attributed plausibly to the sharply increased spending alone and that the evidence 'strongly suggests that targets made a marked difference in reported performance, at least in some important cases', but that 'fragmentary data and all of the familiar problems of identifying causality make it hard to answer such questions definitively'; the subsequent quasi-experimental literature only partially answers this for specific health-care targets. (2) Real versus reported improvement: to what extent do the reported performance improvements reflect real underlying improvement — that is, what is the true prevalence and distribution of gaming behaviors (ratcheting, threshold-chasing and output distortion) across delivery organizations such as hospitals, primary-care practices, ambulance services and local offices, and how can gaming be measured robustly and independently of the provider-reported performance data that gaming itself contaminates? The study found that all three gaming types 'were identified by these interviewees... albeit to highly varying extents' and the reported performance data 'far from problem free', but interviewees varied widely in how much gaming they thought was going on and such estimates were 'hard to assess'. (3) Anti-gaming design: how can target-and-performance-indicator systems be fine-tuned or game-proofed — which of the antidotes the study enumerates (tightening rules and data definitions; modifying targets or key performance indicators; sporadic investigations by public auditors or other oversight bodies; threats of discipline; mystery shoppers; third-party collection of data in the form of user-satisfaction surveys rather than data directly collected by the organizations subject to the targets; information-system architecture limiting creative responses to data entry; converting achieved targets into permanent threshold standards policed by quality overseers) check ratchet, threshold and output-distortion gaming when applied singly or in combination, given the study's observation that many antidotes 'tended to focus on limiting opportunities to game the system... rather than changing motivation'?

Two further points are background only, not part of the problem: On weak verification: the study noted that the slow and half-hearted approach to developing independent verification of performance data 'might itself be interpreted as a form of gaming by the central managers' (the Nelson's-blind-eye reading, a deliberate tolerance of favourable numbers) or, in a more innocent interpretation, simply represent the priorities of a central government in a hurry to show results without imposing extra regulatory burdens on public services — and stated that 'we do not have the evidence to assess which of these two interpretations is correct, and indeed, hard evidence on that issue may never be available'; its pointed observation is that, whichever explanation is correct, remarkably little verification was done from the outset. On regime durability: the study asked only 'what remains to be seen' — whether the target system would last or be replaced by choice-based mechanisms — and itself grounded the continuing importance of the game-proofing question in the fact that, to the extent choice cannot fix everything, the question of how to fine-tune or game-proof target-and-performance-indicator systems remains important.

References

  1. Hood C (2006). Gaming in Targetworld: The Targets Approach to Managing British Public Services. Public Administration Review 66(4):515-521. DOI: 10.1111/j.1540-6210.2006.00612.x. https://doi.org/10.1111/j.1540-6210.2006.00612.x
  2. Bevan G, Hood C (2006). What's Measured Is What Matters: Targets and Gaming in the English Public Health Care System. Public Administration 84(3):517-538. DOI: 10.1111/j.1467-9299.2006.00600.x. https://doi.org/10.1111/j.1467-9299.2006.00600.x
  3. Alvarez-Rosete A, Bevan G, Mays N, Dixon J (2005). Effect of diverging policy across the NHS. BMJ 331(7522):946-950. DOI: 10.1136/bmj.331.7522.946. https://doi.org/10.1136/bmj.331.7522.946
  4. Propper C, Sutton M, Whitnall C, Windmeijer F (2008). Did 'Targets and Terror' Reduce Waiting Times in England for Hospital Care? The B.E. Journal of Economic Analysis & Policy 8(2). DOI: 10.2202/1935-1682.1863. https://doi.org/10.2202/1935-1682.1863
  5. Bevan G, Hamblin R (2009). Hitting and Missing Targets by Ambulance Services for Emergency Calls: Effects of Different Systems of Performance Measurement within the UK. Journal of the Royal Statistical Society Series A 172(1):161-190. DOI: 10.1111/j.1467-985X.2008.00557.x. https://doi.org/10.1111/j.1467-985X.2008.00557.x
  6. Elkomy S, Cookson G (2020). Performance Management Strategy: Waiting Time in the English National Health Services. Public Organization Review 20(1):95-112. DOI: 10.1007/s11115-018-0425-7. https://doi.org/10.1007/s11115-018-0425-7
  7. Jones P, Schimanski K (2010). The four hour target to reduce emergency department 'waiting time': A systematic review of clinical outcomes. Emergency Medicine Australasia 22(5):391-398. DOI: 10.1111/j.1742-6723.2010.01330.x. https://doi.org/10.1111/j.1742-6723.2010.01330.x
  8. Doran T, Fullwood C, Gravelle H, Reeves D, Kontopantelis E, Hiroeh U, Roland M (2006). Pay-for-Performance Programs in Family Practices in the United Kingdom. New England Journal of Medicine 355(4):375-384. DOI: 10.1056/NEJMsa055505. https://doi.org/10.1056/NEJMsa055505
  9. Gravelle H, Sutton M, Ma A (2010). Doctor Behaviour under a Pay for Performance Contract: Treating, Cheating and Case Finding? The Economic Journal 120(542):F129-F156. DOI: 10.1111/j.1468-0297.2009.02340.x. https://doi.org/10.1111/j.1468-0297.2009.02340.x
  10. Campbell S, Hannon K, Lester H (2011). Exception reporting in the Quality and Outcomes Framework: views of practice staff – a qualitative study. British Journal of General Practice 61(585):e183-e189. DOI: 10.3399/bjgp11X567117. https://doi.org/10.3399/bjgp11X567117
  11. Courty P, Marschke G (2004). An Empirical Investigation of Gaming Responses to Explicit Performance Incentives. Journal of Labor Economics 22(1):23-56. DOI: 10.1086/380402. https://doi.org/10.1086/380402
  12. Quinn L, Bird P, Lilford R (2025). Four-Hour Emergency Department Target Performance in England, 2011-2025: A Statistical Process Control and Interrupted Time Series Analysis. SSRN preprint. DOI: 10.2139/ssrn.5803152. https://doi.org/10.2139/ssrn.5803152
  13. Bevan G (2009). Have targets done more harm than good in the English NHS? No. BMJ 338:a3129. DOI: 10.1136/bmj.a3129. https://doi.org/10.1136/bmj.a3129
  14. Gubb J (2009). Have targets done more harm than good in the English NHS? Yes. BMJ 338:a3130. DOI: 10.1136/bmj.a3130. https://doi.org/10.1136/bmj.a3130
  15. Benaine SL (2020). Performance gaming: a systematic review of the literature in public administration and other disciplines with directions for future research. International Journal of Public Sector Management 33(5):497-517. DOI: 10.1108/IJPSM-07-2019-0191. https://doi.org/10.1108/IJPSM-07-2019-0191
  16. Hood C (2007). What happens when transparency meets blame-avoidance? Public Management Review 9(2):191-210. DOI: 10.1080/14719030701340275. https://doi.org/10.1080/14719030701340275
  17. Hood C, Jennings W, Copeland P (2016). Blame Avoidance in Comparative Perspective: Reactivity, Staged Retreat and Efficacy. Public Administration 94(2):542-562. DOI: 10.1111/padm.12235. https://doi.org/10.1111/padm.12235
  18. Hood C, Piotrowska B (2021). Goodhart's Law and the Gaming of UK Public Spending Numbers. Public Performance & Management Review 44(2):250-271. DOI: 10.1080/15309576.2020.1749092. https://doi.org/10.1080/15309576.2020.1749092
  19. Hood C, Piotrowska B (2024). Mature NPM. In: The Resilience of New Public Management, pp. 49-69. Oxford University Press. DOI: 10.1093/oso/9780198883814.003.0002. https://doi.org/10.1093/oso/9780198883814.003.0002
  20. Gao J, Chan HS, Yang K (2021). Gaming in Performance Management Reforms and Its Countermeasures: Symposium Introduction. Public Performance & Management Review 44(2):243-249. DOI: 10.1080/15309576.2021.1894187. https://doi.org/10.1080/15309576.2021.1894187
  21. Caffrey L, Ferlie E, McKevitt C (2019). The strange resilience of new public management: the case of medical research in the UK's national health service. Public Management Review 21(4):537-558. DOI: 10.1080/14719037.2018.1503702. https://doi.org/10.1080/14719037.2018.1503702
  22. O'Dowd A (2020). NHS England proposes new indicators to replace four hour target in emergency departments. BMJ 371:m4855. DOI: 10.1136/bmj.m4855. https://doi.org/10.1136/bmj.m4855
  23. O'Dowd A (2025). NHS England cuts targets in drive to reduce waiting times. BMJ 386:r235. DOI: 10.1136/bmj.r235. https://doi.org/10.1136/bmj.r235