Public services pursue multidimensional goals — quality, equity, access, responsiveness, and long-term outcomes — that performance-measurement regimes compress into measurable indicators. Wherever stakes are attached to such indicators, recurring dysfunctions follow: gaming, tunnel-vision output distortion of unmeasured dimensions, threshold-chasing, indicator proliferation with cross-level inconsistency, and performance paradoxes in which measured performance rises while true effectiveness does not. The standard policy response is iterative redesign: adding, redefining, or replacing indicators after each observed manipulation. Whether any such redesign sequence can restore measurement validity, or whether dysfunction is a persistent structural property of proxy measurement under stakes in multidimensional public services, remains unsettled. The sources raising the question (Lapsley 2008; Bevan and Hood 2006; McLean, Haubrich and Gutierrez-Romero 2007; Micheli and Neely 2010; Meijer 2007) conclude that persistence has been observed without identifying conditions under which redesign would restore validity. Later literature documents gaming persisting into the 2020s (Tenbensel et al. 2019; Quinn, Bird and Lilford 2024), proposes untested motivation-and-opportunity frameworks (Taylor 2020), and continues to list gaming as an unresolved challenge in reviews (Gao 2015; Muller 2018). The sources leave unresolved whether any redesign sequence can restore measurement validity, including any regime for which none does, and which conditions on regime design govern whether iterative indicator substitution converges to valid measurement, displaces distortion into newly unmeasured dimensions, or leaves it unchanged.

Background

Since the 1980s, many governments have sought to make public services accountable by importing private-sector management techniques, a reform wave known as New Public Management (NPM). A core NPM instrument is performance measurement: replacing broad, partly qualitative judgments about multidimensional service goals with quantitative performance indicators — measurable proxies such as waiting times, exam results, or inspection scores — and publishing them, sometimes with rewards and sanctions attached.

A long research line documents systematic dysfunctions when multidimensional goals are compressed into measured proxies. Campbell (1979) stated the general hazard now known as Campbell's law: the more any quantitative social indicator is used for decision-making, the more subject it becomes to corruption pressures and the more it distorts the processes it monitors. Smith (1995) catalogued the behavioral pathologies of publishing performance data in the public sector: tunnel vision (emphasis on what is measured at the expense of what is not), suboptimization, measure fixation, misrepresentation, and gaming — deliberately manipulating measured performance rather than improving the underlying service. Strathern (1997) condensed the logic into the aphorism that when a measure becomes a target it ceases to be a good measure (the public-administration version of what is often called Goodhart's law), and van Thiel and Leeuw (2002) describe the resulting performance paradox: indicator values improve while the underlying effectiveness of services does not. On the theory side, the multitask principal-agent model of Holmstrom and Milgrom (1991) — analyzing an agent whose effort is split across several tasks when only some tasks are measurable — shows formally that high-powered incentives on the measurable task distort effort away from unmeasurable tasks, supplying an economic mechanism for why these dysfunctions should be expected whenever stakes are attached to incomplete measures.

The empirical record is extensive. Bevan and Hood (2006) showed that the star-rating regime for the English National Health Service (NHS) of 2000-2004 produced large measured improvements alongside well-documented gaming — holding patients in ambulances outside emergency departments to stop the four-hour waiting-time clock, spikes of reported performance exactly at target thresholds, and reclassification of ambulance response-time data — with no systematic independent audit of the underlying data. McLean, Haubrich and Gutierrez-Romero (2007) documented analogous distortions in the Comprehensive Performance Assessment (CPA), the Audit Commission's regime that converted hundreds of indicators for English local authorities into categorical ratings: measured indicators rose while unmeasured service dimensions deteriorated (output distortion), managers targeted threshold-driven quick wins, and scores could be raised by discretionary spending. Micheli and Neely (2010) traced the English attempt to cascade Public Service Agreement targets from central government to local delivery and found not a coherent 'golden thread' but indicator multiplication and cross-level inconsistency — a small set of national indicators expanding into dozens of weakly corresponding local ones. Meijer (2007) showed that publishing limited indicator sets on the Internet (Dutch school 'Quality Cards' and hospital 'Waiting List Monitors') fostered strategic behavior — grade inflation, creaming and dumping of patients — with substantial risk of a performance paradox.

The standard policy response to gaming has been iterative redesign of the measurement system itself: adding indicators to cover gamed dimensions, redefining or replacing them, reweighting composite ratings, or replacing whole regimes with successors. Lapsley (2008), reviewing the trajectory of NPM, concluded that performance measurement in multidimensional public services will continue to produce dysfunctional and contentious effects — comparability problems, manipulability, perverse incentives — which iterative redesigns have not resolved and which are expected to persist. Whether that persistence is structural — an unavoidable consequence of compressing multidimensional goals into measurable proxies under stakes — or whether identifiable design conditions exist under which iterative redesign converges to valid measurement, is not settled. Bevan and Hood (2006) proposed mitigating redesigns (less predictable monitoring mechanics, systematic independent data audit) while stating flatly that gaming cannot be eliminated. Later studies show gaming varies across organizations and time — Tenbensel et al. (2019) found emergency-department target gaming in New Zealand common but unevenly distributed, growing where the opportunity to game (absence of independent verification) expanded; Terman and Yang (2016) and Taylor (2020) argue gaming depends jointly on officials' motivation and contextual opportunity — but these are untested frameworks rather than validated conditions. Recent reviews (Gao 2015; Van Dooren, Bouckaert and Halligan 2015; Muller 2018) continue to describe gaming as a remaining, unresolved challenge, and Koretz (2008) shows the same persistence for high-stakes educational testing despite repeated test redesigns. Adjacent formal work taxonomizing Goodhart's-law variants in machine-learning alignment (Manheim and Garrabrant 2018) formalizes proxy divergence but supplies no conditions for public-service measurement regimes.

Problem Statement

Can the iterative substitution and redesign of measurable performance indicators for multidimensional public-service goals ever restore measurement validity — that is, bring about a measurement regime in which movements of the published indicators reliably track the underlying multidimensional goals rather than gaming, output distortion of unmeasured dimensions, or a performance paradox — or is such dysfunction a persistent structural property of proxy measurement under stakes in multidimensional public services?

Here 'measurement validity' is understood in the field's sense: correspondence between indicator movements and the true attainment of the service's multidimensional goals, including the unmeasured dimensions, and not merely internal consistency among the measured indicators. 'Iterative redesign' covers the levers the sources raising the question themselves discuss for countering gaming, when such changes are made in response to observed gaming or distortion: adding indicators to cover gamed dimensions, redefining or replacing indicators, reweighting composite scores or categorical ratings, and wholesale replacement of one measurement regime by its successor; changes in the predictability of monitoring and assessment mechanics; changes in independent verification and audit arrangements for the reported data; and changes in the stakes (rewards, sanctions, financial penalties) attached to targets. The structural-persistence branch of the question concerns regimes in which targets carry such stakes.

The unresolved question is which conditions on regime design govern whether sequences of redesigns converge toward valid measurement, displace dysfunction into newly unmeasured dimensions, or leave it unchanged, including any regime for which no redesign sequence restores validity. The conditions at issue, whether formal or empirically grounded, include incentive intensity and the stakes attached to indicators; dimensionality coverage and composition of the indicator set; independent verification and audit capacity; participation and target-setting processes; predictability of monitoring; and stability versus churn of the regime. Whether evidence from at least one real, iteratively redesigned public-service measurement regime distinguishes genuine improvement from gaming, and thereby bears on whether and when iterative redesign restores measurement validity, leaves it unchanged, or displaces the dysfunction, is part of the same open question.

References

  1. Irvine Lapsley (2008). 'The NPM Agenda: Back to the Future.' Financial Accountability & Management 24(1): 77-96. DOI: 10.1111/j.1468-0408.2008.00444.x, https://doi.org/10.1111/j.1468-0408.2008.00444.x
  2. Pietro Micheli and Andy Neely (2010). 'Performance Measurement in the Public Sector in England: Searching for the Golden Thread.' Public Administration Review 70(4): 591-600. DOI: 10.1111/j.1540-6210.2010.02180.x, https://doi.org/10.1111/j.1540-6210.2010.02180.x
  3. Iain McLean, Dirk Haubrich and Roxana Gutierrez-Romero (2007). 'The Perils and Pitfalls of Performance Measurement: The CPA Regime for Local Authorities in England.' Public Money and Management 27(2): 111-118. DOI: 10.1111/j.1467-9302.2007.00566.x, https://doi.org/10.1111/j.1467-9302.2007.00566.x
  4. Albert Jacob Meijer (2007). 'Publishing public performance results on the Internet.' Government Information Quarterly 24(1): 165-185. DOI: 10.1016/j.giq.2006.01.014, https://doi.org/10.1016/j.giq.2006.01.014
  5. Peter Smith (1995). 'On the unintended consequences of publishing performance data in the public sector.' International Journal of Public Administration 18(2-3): 277-310. DOI: 10.1080/01900699508525011, https://doi.org/10.1080/01900699508525011
  6. Donald T. Campbell (1979). 'Assessing the impact of planned social change.' Evaluation and Program Planning 2(1): 67-90. DOI: 10.1016/0149-7189(79)90048-X, https://doi.org/10.1016/0149-7189(79)90048-X
  7. Marilyn Strathern (1997). "'Improving ratings': audit in the British university system." European Review 5(3): 305-321. DOI: 10.1002/(SICI)1234-981X(199707)5:3<305::AID-EURO184>3.0.CO;2-4, https://doi.org/10.1002/(SICI)1234-981X(199707)5:3<305::AID-EURO184>3.0.CO;2-4
  8. Bengt Holmstrom and Paul Milgrom (1991). 'Multitask Principal-Agent Analyses: Incentive Contracts, Asset Ownership, and Job Design.' The Journal of Law, Economics, and Organization 7 (Special Issue): 24-52. DOI: 10.1093/jleo/7.special_issue.24, https://doi.org/10.1093/jleo/7.special_issue.24
  9. Gwyn Bevan and Christopher Hood (2006). "What's Measured Is What Matters: Targets and Gaming in the English Public Health Care System." Public Administration 84(3): 517-538. DOI: 10.1111/j.1467-9299.2006.00600.x, https://doi.org/10.1111/j.1467-9299.2006.00600.x
  10. Sandra van Thiel and Frans L. Leeuw (2002). 'The Performance Paradox in the Public Sector.' Public Performance & Management Review 25(3): 267-281. DOI: 10.1080/15309576.2002.11643661, https://doi.org/10.1080/15309576.2002.11643661
  11. Jie Gao (2015). 'Performance Measurement and Management in the Public Sector: Some Lessons from Research Evidence.' Public Administration and Development 35(2): 86-96. DOI: 10.1002/pad.1704, https://doi.org/10.1002/pad.1704
  12. Wouter Van Dooren, Geert Bouckaert and John Halligan (2015). Performance Management in the Public Sector. Routledge. DOI: 10.4324/9781315817590, https://doi.org/10.4324/9781315817590
  13. Daniel Koretz (2008). Measuring Up: What Educational Testing Really Tells Us. Harvard University Press (online ed. 2009). DOI: 10.2307/j.ctv1503gxj, https://doi.org/10.2307/j.ctv1503gxj
  14. Jerry Z. Muller (2018). The Tyranny of Metrics. Princeton University Press. DOI: 10.23943/9781400889433, https://doi.org/10.23943/9781400889433
  15. Christopher Hood and Ruth Dixon (2015). A Government that Worked Better and Cost Less? Oxford University Press. DOI: 10.1093/acprof:oso/9780199687022.001.0001, https://doi.org/10.1093/acprof:oso/9780199687022.001.0001
  16. Jessica N. Terman and Kaifeng Yang (2016). 'Reconsidering Gaming in an Accountability Relationship: The Case of Minority Purchasing in Florida.' Public Performance & Management Review 40(2): 281-309. DOI: 10.1080/15309576.2016.1177560, https://doi.org/10.1080/15309576.2016.1177560
  17. Jeannette Taylor (2020). "Public Officials' Gaming of Performance Measures and Targets: The Nexus between Motivation and Opportunity." Public Performance & Management Review 44(2): 272-293. DOI: 10.1080/15309576.2020.1744454, https://doi.org/10.1080/15309576.2020.1744454
  18. Tim Tenbensel, Peter Jones, Linda Maree Chalmers, Shanthi Ameratunga and Peter Carswell (2019). "Gaming New Zealand's Emergency Department Target: How and Why Did It Vary Over Time and Between Organisations?" International Journal of Health Policy and Management. DOI: 10.15171/ijhpm.2019.98, https://doi.org/10.15171/ijhpm.2019.98
  19. Lisa M. Lines (2020). "Games People Play: Lessons on Performance Measure Gaming from New Zealand. Comment on 'Gaming New Zealand's Emergency Department Target: How and Why Did It Vary Over Time and Between Organisations?'" International Journal of Health Policy and Management. DOI: 10.34172/ijhpm.2020.41, https://doi.org/10.34172/ijhpm.2020.41
  20. Laura Quinn, Paul Bird and Richard Lilford (2024). 'Effect of cancer waiting time standards in the English National Health Service: a threshold analysis.' BMC Health Services Research. DOI: 10.1186/s12913-024-11350-z, https://doi.org/10.1186/s12913-024-11350-z
  21. David Manheim and Scott Garrabrant (2018). 'Categorizing Variants of Goodhart's Law.' arXiv:1803.04585, https://arxiv.org/abs/1803.04585