The Global Party Survey (GPS, fieldwork 2019) used 1,861 country experts to place 1,043 political parties in 163 countries on 0-10 scales, including a core Pluralism-Populism scale, producing the broadest expert-based measure of party populist rhetoric available. The source study itself posed the unfinished scientific problem: whether this approach can yield consistent, valid, and reliable measures of party populism worldwide. Three linked components of that problem remain open as of 2026: (1) temporal reliability, since the announced second wave (GPS-2023) has been fielded but no dataset or publication has been released, so no over-time stability check of the populism estimates exists; (2) estimate precision where few experts responded, notably in small developing societies and closed autocracies, which the source reports yield around five expert judgments per party versus around nineteen in liberal democracies and correspondingly large confidence intervals; and (3) independent cross-validation of the populism scores against alternative measures beyond the original robustness checks. The question is live because the GPS is heavily used as an input in comparative politics, while European instruments (POPPA, the PopuList) and text-based or machine-learning indices have advanced measurement only regionally or remain contested.

Background

Populism in this literature is a thin, cross-cutting feature of party communication: claims that legitimate political authority rests with a unitary 'will of the people' and that self-serving elites have usurped it, rather than a substantive program on the economy or immigration. Measuring it for parties across countries has followed four routes. First, expert surveys, in the tradition of the Chapel Hill Expert Survey (CHES, an expert survey of party positioning run by a European academic consortium): Pippa Norris's Global Party Survey (GPS, fieldwork November-December 2019) asked 1,861 party and election experts to place 1,043 parliamentary parties in 163 countries on 21 core items, each a ten-point scale. Its core populism item places each party on a 0-10 Pluralism-Populism scale (0 = strongly favors pluralist rhetoric, accepting that elected leaders govern constrained by minority rights, compromise, and checks and balances; 10 = strongly favors populist rhetoric, challenging the legitimacy of established institutions in the name of the people's will), supplemented by five further populist-rhetoric items (salience of populist appeals, whether the people or leaders should decide, whether politicians should lead or follow the people, perceived honesty of politicians, and respect for liberal-democratic norms and checks on executive power); parties scoring above 7.5 on the core scale were labeled 'strongly populist' in the study's descriptive typology. Second, text-based measurement, exemplified by holistic grading, in which trained coders read a leader's speeches and assign a single 0-2 populism score (Hawkins 2009), extended globally by the Global Populism Project for over 200 presidents and prime ministers. Third, expert-informed classification lists such as the PopuList, which classifies European parties as populist, far-left, or far-right using expert consultation, and the Populism and Political Parties Expert Survey (POPPA), which measures party populism in Europe as a five-item multidimensional latent construct. Fourth, automated text analysis of party manifestos using supervised machine learning. The GPS paper validated its estimates through correlations with CHES, the Parliaments and Governments database, the Comparative Manifesto Project, the first PopuList, and party-voter locations in the World/European Values Surveys, but it also documented the design's unresolved weaknesses: expert coverage is thin exactly where populism research is expanding (around five experts per party in closed autocracies versus around nineteen in liberal democracies), the underlying party lists go stale in volatile party systems ('flash' parties, unstable coalitions), many candidate items had to be dropped to keep the questionnaire answerable, and single-wave estimates cannot demonstrate temporal reliability. The source paper's conclusion therefore states that the dataset 'still needs much further refinement in future iterations': improving party-list accuracy, gathering more expert assessments in the hardest cases, replicating the approach for political leaders as well as parties, and using subsequent survey waves as reliability checks. These components remain open at the GPS's global scope as of 2026, while methodological critiques have sharpened doubts about whether any current instrument - expert-survey or automated - validly measures populism across countries and over time.

Problem Statement

The problem posed by the source study is to establish whether the Global Party Survey's expert-based approach can measure party populist rhetoric in ways that are consistent, valid, and reliable across countries and over time. Concretely, the following jointly linked components serve this single measurement objective: (1) temporal reliability - establish the over-time stability of the Pluralism-Populism estimates for common parties from a replicated expert-survey wave with continuity of items and experts relative to GPS-2019, documenting the continuity of items and experts and the party coverage on which that evidence rests; (2) estimate precision in hard cases - reduce the uncertainty of party populism estimates in countries where few experts responded (smaller developing societies and closed autocracies, where the first wave averaged around five expert judgments per party), beyond the user-side remedies already documented in the source (the Min_experts filter dropping parties with fewer than 4.9 experts, weighting by expert familiarity and reported difficulty, and cross-country aggregation), together with the party-list refinements that greater precision requires in volatile party systems ('flash' parties and unstable coalitions); and (3) independent cross-validation - evaluate the GPS populism scores against alternative measures (expert-informed classifications such as the PopuList, multidimensional expert surveys such as POPPA, and text-based or machine-learning indices) beyond the robustness checks reported in the 2020 study, quantifying agreement and diagnosing divergences. The source also suggests extending the approach to other actor types, in particular political leaders, as part of testing the generality of expert-survey measurement of populist rhetoric; this extension is advisory and is not required for a complete answer. The question is evaluative: for any one component, a well-evidenced negative finding - for example, that the Pluralism-Populism estimates are not stable across waves, or that precision in thin-coverage autocracies cannot be brought within usable bounds - establishes the limits of trustworthiness for that component; such a finding does not by itself resolve the other two. The objective is to establish the conditions under which the GPS-style global expert survey of party populism is trustworthy - including where its coverage is thin and where its populism scale diverges from alternative operationalizations.

References

  1. Pippa Norris (2020). 'Measuring populism worldwide.' Party Politics 26(6): 697-717. DOI: 10.1177/1354068820927686. https://doi.org/10.1177/1354068820927686
  2. Maurits J. Meijers, Andrej Zaslove (2021). 'Measuring Populism in Political Parties: Appraisal of a New Approach.' Comparative Political Studies 54(2): 372-407. DOI: 10.1177/0010414020938081. https://doi.org/10.1177/0010414020938081
  3. Andrej Zaslove, Robert A. Huber, Maurits J. Meijers (2025). 'The state of populism: Introducing the 2023 wave of the Populism and political parties expert survey.' Party Politics. DOI: 10.1177/13540688251361813. https://doi.org/10.1177/13540688251361813
  4. Matthijs Rooduijn, Andrea L. P. Pirro, Daphne Halikiopoulou, Caterina Froio, Stijn Van Kessel, Sarah L. De Lange, Cas Mudde, Paul Taggart (2024). 'The PopuList: A Database of Populist, Far-Left, and Far-Right Parties Using Expert-Informed Qualitative Comparative Classification (EiQCC).' British Journal of Political Science 54(3): 969-978. DOI: 10.1017/S0007123423000431. https://doi.org/10.1017/S0007123423000431
  5. Jose Javier Olivas Osuna, Jose Rama (2022). 'Recalibrating populism measurement tools: Methodological inconsistencies and challenges to our understanding of the relationship between the supply- and demand-side of populism.' Frontiers in Sociology 7: 970043. DOI: 10.3389/fsoc.2022.970043. https://doi.org/10.3389/fsoc.2022.970043
  6. Kirk A. Hawkins (2009). 'Is Chávez Populist? Measuring Populist Discourse in Comparative Perspective.' Comparative Political Studies 42(8): 1040-1067. DOI: 10.1177/0010414009331721. https://doi.org/10.1177/0010414009331721
  7. Erin K. Jenne, Kirk A. Hawkins, Bruno Castanho Silva (2021). 'Mapping Populism and Nationalism in Leader Rhetoric Across North America and Europe.' Studies in Comparative International Development 56(2): 170-196. DOI: 10.1007/s12116-021-09334-9. https://doi.org/10.1007/s12116-021-09334-9
  8. Semir Dzebo, Erin K. Jenne, Levente Littvay, Kirk A. Hawkins, Olaf van der Veen (2026). 'US governors populism database: Assessing the impact of Donald Trump on state-level discourse.' Party Politics 32(2): 300-307. DOI: 10.1177/13540688251327564. https://doi.org/10.1177/13540688251327564
  9. Jessica Di Cocco, Bernardo Monechi (2022). 'How Populist are Parties? Measuring Degrees of Populism in Party Manifestos Using Supervised Machine Learning.' Political Analysis 30(3): 311-327. DOI: 10.1017/pan.2021.29. https://doi.org/10.1017/pan.2021.29
  10. Michael Jankowski, Robert A. Huber (2023). 'When Correlation Is Not Enough: Validating Populism Scores from Supervised Machine-Learning Models.' Political Analysis 31(4): 591-605. DOI: 10.1017/pan.2022.32. https://doi.org/10.1017/pan.2022.32
  11. Andrea Celico, Martin Rode, Ignacio Rodriguez-Carreno (2024). 'Will the real populists please stand up? A machine learning index of party populism.' European Journal of Political Economy 82: 102529. DOI: 10.1016/j.ejpoleco.2024.102529. https://doi.org/10.1016/j.ejpoleco.2024.102529
  12. Bastian Herre (2023). 'Identifying Ideologues: A Global Dataset on Political Leaders, 1945-2020.' British Journal of Political Science 53(2): 740-748. DOI: 10.1017/S0007123422000217. https://doi.org/10.1017/S0007123422000217
  13. Nicolas de la Cerda, Ryan Bakker, Seth Jolly, Jonathan Polk, Ruth Dassonneville, Jelle Koedam, Patrick Leslie, Jill Sheppard, Roi Zur (2025). 'A Global Scale of Economic Left-Right Party Positions: Cross-National and Cross-Expert Perceptions of Party Placements.' The Journal of Politics 87. DOI: 10.1086/736578. https://doi.org/10.1086/736578
  14. Till O. Weber (2022). 'Quantitative Populism: Of Shapes and Shades.' Polity 54(3): 519-528. DOI: 10.1086/719796. https://doi.org/10.1086/719796